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I.   INTRODUCTION 


In  a  recent  SHARE  study  the  annual  growth  rate  of 
software  aemands  over  the  years  1R57  -  1985  was  estimated  to 
be  as  high  as  21-23%  (15],  The  same  study  indicates  that 
software  production  may  develop  at  a  growth  rate  of  only 
1 1  .b-17%. 

Besides  this  quant i  tat ive  aao,  Roehm  [7]  ooints  out  the 
following  aualitative  deficiencies:  Since  software  is  in- 
creasingly used  for  important  functions  such  as  defense  sys- 
tems* traffic  control  systems*  and  medical  purposes* 
software  is  needed  which  can  be  trusted.  Although  80%  of 
the  money  used  for  the  production  of  the  Apollo  Vanned 
Spaceflight  Program  was  devoted  to  testing  [121,  software 
errors  occurred  in  Aoollo  8,  11/  and  1  '4  .  One  critical  error 
even  occurred  during  the   lunar  landing  phase. 

Software  engineering  (for  definition  see  Chapter  II)  may 
be  accepted  as  the  method  which  yields  the  answer  to  the 
problem  of  producing  reliable  software  at  reasonaole  cost 
[71.  Little  is  known  about  software  errors  besides  their 
existence  and  their  negative  influence  on  cost  and  perfor- 
mance. Increasing  difficulties  to  produce  sufficient 
amounts  of  hiqh  Quality  software  make  it  mandatory  to  invent 
software  development  methods  which  are  designed  to  avoid  the 
most   common   software   errors*   achieve   easier  testing  and 
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debugging  and  improve  methods  of  error  detection. 

This  cannot  be  aone  without  detailed  studies  about  the 
occurrence  of  software  errors.  Only  this  knowledge  will 
help  to  identify  the  nature  of  errors  ana  finally  lead  to 
improvements  of  software  methodoloay.  Other  motivating  fac- 
tors for  this  study  were: 

1.  Lack  of  currently  available  data  to  support   analysis 

and   research   in   the   area   of  error  ocurrence  and 
detection. 

2.  The  need  to  examine   existing  software  error  models. 

The  objective  of  the  experiment  was  to  gather  accurate 
and  complete  information  about  errors*  their  occurrence  and 
detection  throughout  the  entire  orocess  of  software  develop- 
ment. Limitations  of  the  experiment  were  the  relatively 
small  number  of  projects  and  trie  lack  of  exoerienced  pro- 
grammers. The  use  of  the  experimenter  as  the  subject  pro- 
grammer and  evaluator  introduced  biases  also.  The  scope  of 
the  experiment  did  not  include  the  program  maintenance 
phase.  It  was  felt  to  be  of  great  importance  to  record  all 
errors/  even  trivial  errors  which  could  hurt  the  image  of 
the  programmer.  A  detailed  description*  of  the  experiment 
and  the  error  data  is  given  in  chapters  Q  thru  5  and  in  the 
append  ices. 
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II.   CONCEPTS  AND  KEYWORDS 

Communication  among  computer  scientists  is  hindered  by 
the  lack  of  common  terminoloay.  In  order  to  avoid  misunder- 
standing, the  most  important  keywords  will  be  defined  in 
this  chanter.  wherever  possible  definitions  are  adopted 
from  other  publications. 

A.   SOFTWARE  ENGINEERING  TERMS 

1.  Software  Engineering 

"The  practical  application  of  scientific  knowledge 
in  the  design  and  construction  of  comouter  programs  and 
the  associated  documentation  reauired  to  develop* 
operate  and  maintain  them".  T7] 

2.  Top-Down  Design 

A  program  design  method  which  starts  at  a  very  gen- 
eral level  Dy  identifying  major  functions  and  oroceeds 
stepwise  to  lower  levels  to  the  identification  of  lesser 
functions  that  derive  from  the  major  ones.  "Too-down 
design"  and  "hierarchical  orogram  design"  are  equivalent 
terms.  [121 
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3.   Structured  Programming 

Although  this  term  is  used  in  numerous  Publications 
its  understanding  varies  because  of  its  conceptual  na- 
ture. As  oroposea  by  Dijkstra  as  early  as  19b5  [30],  it 
first  was  a  conceot  to  eliminate  GO  TO  statements. 
Later  on  it  was  combined  with  the  idea  of  top-down 
design.   [31) 

Rather  than  providing  a  formal  definition,  Yourdan 
[12]  and  Auerbach  [27]  describe  structured  programming 
oy  its  objectives:  "increase  of  readability"  and  "de- 
crease of  testing  problems."  The  following  definition 
of  structured  programming  as  it  applies  to  this  paper  is 
derived  from  K i re hgaessner  '  s  description.   [2b] 

Structured  programming  is  a  method  of  developing 
programs  for  whicn  the  underlying  concept  is  a  top-down 
design  which  eventually  leads  to  a  modular  structure  of 
the  program.  All  algorithms  are  designed  using  the  five 
major  constructs  allowed  in  structured  programming: 

-  Sequential  statement 

-  IF  THEN  ELSE  statement 

-  WHILE  loop  (conditional  loop) 

-  FOR  loop  (iterative  loop) 

-  CASE  statement 

In   addition   to   this   definition   it    snould    De 
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emphasized  that  structured  programming  is  a  aesign  dis- 
cipline rather  than  a  way  of  writing  code.  Therefore 
it  should  not  be  defined  as  "coding  without  GO  TO 
statements."  Methods  have  been  developed  to  implement 
structured  designs  in  FORTRAN.  [12]  However/  it  is 
easier,  ana  more  readable  to  use  a  block  structured 
lanquaqe/  such  as  ALGOL*  to  imolement  a  structured 
design. 

4 .  D  i  rec  t  ed  Graph 

"A  directed  graph  is  a  geometric  graph*  consisting 
of  nodes  and  arcs  with  a  direction  of  traversal  associ- 
ated with  each  arc."  [16] 

5 .  Modu 1 e 

"A  module  is  a  ohysical  comoinaticn  of  program 
instructions  that  is  independent  of  others  with  rescect 
to  compiling,  assembling  and  loading  and  which  performs 
a  specific  function."  [16]  "Each  module  has  a  small 
number  of  interactions  with  other  modules."  I  ^  ] 

6  .   H r oar am 

A  program  is  a  set  of  integrated  modules.   [lb] 

7.   Software  Development  Pmcess 

The   software   develooment   process   includes    the 
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f o 1 1 ow  i  ng  st ages : 

-  Problem  Analysis 

-  Design  and  Design  Review 

-  Codi  nq 

-  Debugg i  nq 

-  Testing 

-  In  t eorat  i  on 

-  Imo 1 emen  t  a  t  i  on 

-  Maintenance 

Desoite  minor  differences  with  other  publications 
such  as  fiuerbach  [21  the  above  view  of  the  software 
deve  1  oofnen t  Drocess  *ill  be  adopted  as  the  aefinition  of 
the  software  development  Drocess  for  this  thesis. 
Althouah  the  development  staaes  are  listea  sequentially 
it  should  be  noted  that  software  production  in  general 
is  viewed  as  a  dynamic  process  including  feedbacks  to 
earlier  stages  of  a  project.  Thus  none  of  these  phases 
may  be  considered  to  be  completed  when  the  subseauent 
phase  has  starteo.  In  some  projects/  testing  itself  is 
preceded  by  activities  such  as  the  writing  of  test 
specifications  anc  test  procedures.  Fiaure  1  indicates 
nossiple  feedback  situations  within  a  complex  program 
development  orocess. 
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*  IMPLEMENTATION   o 

FIGURE  1 
THE  FEEDRACK  CYCLES  DURING  A  SOFTWARE  DEVELOPMENT  PROJECT 


8 .   Common  Data 

Common   Data   is   data   which   is   used   by   several 
modu 1 es . 


-  14  - 


B.   TESTING,  DEBUGGING  AND  ERRORS 

1 .   Debuggi  ng 

(a)  Debugging  is  the  action  to  check  subroutines  and 
modules  as  to  whether  or  not  they  Derform  according 
to  orogrammer  exoectat  ions.  It  is  also  the  action 
which  one  takes  to  locate  and  correct  known  errors. 

(b)  Testing  is  the  action  taken  after  modules  have 
been  integrated  to  check  whether  or  not  the  program 
meets  specifications.  As  stated  by  Dijkstra  [29]  , 
testing  can  only  establish  the  presence  of  errors, 
not  their  absence . 

<?.   Software  Error 

A  software  error  is  a  mistake  made  during  software 
development,  which  leaas  to  an  incorrect  action  or 
result  with  resoect  to  the  proaram  soec i f i c a t i ons  within 
the  program.  (loj  Errors  made  in  the  specification 
phase  of  the  program,  hardware  errors  and  compiler  er- 
rors (i.e.  errors  within  the  compiler  program)  were  not 
studied  durina  this  exoeriment.  Categorization  and 
definitions  of  software  errors  with  respect  to  their 
occurrence  are    given  in  chaoter  3. 
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C.   SOFTWARE  RELIABILITY  AND  CORRECTNESS 

1.  Software  Reliability 

"Software  reliability  is  the  probability  that  a  com- 
Duter  Drogram  will  oerform  its  (user)  intended  function 
for  a  soec i f i ed  time  interval  under  stated  operating 
conditions."  t  1  7  3 

2.  Software  Correctness 

"The  feature  of  software  that  renders  it  operation- 
ally useful  for  its  intenaed  functions  is  its  correct- 
ness. Correct  software  does  the  thinos  intended  by 
desian."  [9] 
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Ill .   DEFINITION  OF  ERROR  CATEGORIES  AND  TYPES 

When  analyzing  the  causes  of  errors  in  software  produc- 
tion  there  is  an  obvious  need  to  precisely  describe  an  error 
in  accordance  with  its  category  and  tyoe.  One  of  the  objec- 
tives of  this  study  was  to  identify  errors  with  respect  to 
their  occurrences  ana  causes. 

It  was  was  found  that  many  errors  in  software  oroduction 
belong  to  one  of  the  following  major  categories: 

1.  Problem  Specification  Errors 

2 .  System  Design  Errors 

3.  Program  Desian  Errors 

4 .  Codinq  Errors 

5.  Clerical  trrors 
o.  Debugging  Errors 
7  .  Testing  Errors 

w.   Implementation  Errors 

A.   DEFINITONS  OF  ERROR  CATEGORIES 

1.   Problem  Soecification  Error 

Any  mistake  or  deficiency  which  occurs  in  the 
analysis  of  the  proaram  aooli cation  or  in  specifying  the 
software  requirements  with  resoect  to  the  intended  oro- 
qram    aoplication   such   as   incomplete*   erroneous   or 
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ambiguous  statements. 

2.  System  Design  Error 

An  error  maae  in  the  transformation  of  the  user- 
originated  program  snecifi  cations  into  systems  design 
spec  ifications. 

3.  Program  Design  Error 

A  orogram  design  error  is  one  made  in  the  transfor- 
mation of  the  system  design  into  specific  algorithms  ana 
data  st  rue  t  u  res  . 

4.  Coding  Error 

A  coding  error  is  one  which  is  made  during  the 
transformation  of  a  program  design  into  source  language. 

5.  Clerical  Error 

A  clerical  error  is  human  failure  which  is  made  dur- 
ing the  transformation  of  any  ohysical  representation 
(such  as  a  coding  sheet)  of  tne  orogram  or  parts  of  the 
program  to  another  physical  reoresen t a t i on  (such  as 
nunc  h  c  a  rds  )  . 

fa.   Debuaging  Error 

I naoo roo r i a t e  use  of  debugging  tool Sr  insufficient 
or   i naoo rop r i at e   selection  of  test  casesr  test  data  or 
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misinterpretation  of  deouqainq  results. 

7.  Testinq  Error 

Inappropriate  or  insufficient  test  cases  or  test 
data/  misunderstanding  of  functional  requirements  of  the 
program^  deviation  from  test  olan  or  misinterpretation 
of  test  results. 

8.  Implementation  Error 

Unexpected  environmental  problem  which  occurs  during 
the  implementation  of  a  orogram  which  could  not  have 
been  anticipated  in  a  previous  stage  of  the  software 
development.  This  includes  cases  such  as  changes  of 
haraware  by  the  manufacturer  and  performance  problems 
with  operatinq  systems. 

Accorainq  to  the  cefinition  of  software  errors*  an  error 
which  aoes  not  lead  to  an  incorrect  action  or  result  within 
the  program  cannot  be  identified  as  a  software  error.  For 
example  insufficient  test  cases  auring  aebugging  of  a 
subroutine  may  not  lead  to  a  software  error.  However/  the 
potential  exists  for  undetected  software  errors  when  test 
cases  are  insufficient.  According  to  the  definition  of 
debuqgina  errors/  errors  of  this  category  do  not  necessarily 
affect  the  proaram.  Therefore  debugging  errors  will  not  be 
detected   very  often;   however/  because  of  being  a  potential 
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error  source*  errors  of  this  category  were  considered  to  be 
important.  During  the  exDeriment  described  in  chapter  fl  all 
debuqging  errors  were  recorded.  Similarly  design  errors, 
which  were  detected  during  a  design  review, 'or  faulty  com- 
ments were  recorded.  This  was  of  great  importance/  since  it 
yields  significant  information  about  error  sources. 


B.   DEFINITION  OF  ERROR  TYPES 

The  list  of  error  type  definitions  given  below  identi- 
fies the  most  common  types  of  errors  within  the  major  ca- 
teaories  of  software  errors.  If  appropriate,  examples  are 
given  for  clarity. 

These   definitions  are  important   because  it  has  been 

discovered  that  without  a  aefinition  of  error  types,  errors 
of  the  same  tyDe  could  be  recorded  in  different  ways,  not 
only  by  different  o  rog  ramoie  r  s ,  but  also  by  the  same  person 
at  different   times. 

1 .   Design  Errors 

The  following  types  of  errors  apply  to  both  ca- 
teaories  "System  Design  Errors"  and  "Program  Design 
Errors": 

Dl  :   Communication  Error 

An    error   occurs   due   to   improper   communication 
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between  members  of  a   programming   team   or   due   to 
inappropriate   systems  design  specifications.   Typi- 
cal cases  are  conflicting  interpretation   of   common 
data  and  m  i  sunde  rs  t  and  i  no.   of  module  functions. 
D'd     :       Design  Negliqence 

Parts  of  program  or  system  design  spec i f c a t i ons  are 
neglected  or  necessary  cooperation  with  the 
originator(s)  of  the  SDecification  have  been  omit- 
ted/ such  as  neglecting  desired  outout  formats  or 
lack  of  documentation. 
Di  :   Forgotten  Cases  or  Steos 

while  solving  the  problem  one  or  more  cases  have 
not  been  considered,  such  as  neglecting  leap  years 
for  calculation  of  dates  or  necessary  step(s)  to 
solve  the  problem  have  been  omitted/  such  as  deter- 
mining the  status  of  peripheral  devices  before  open- 
ing I/Of  or  forgotten  conversion  from  decimal  to 
binary  or  vice  versa. 
04  :   Timing  Proolems 

Misconceotion   in  scheduling  program  events/  such  as 
assianing  inappropriate  priorities  or  changing   data 
before  completion  of  I/O. 
Ob  :   Errors  in  I/O  Concepts 

Errors  made  in  the  design  of  input  or  output/  such 
as  misunderstanding  of  hardware  reaui  rements  of  I/O 
devices     or     channels.      Typical    cases    are 
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misunderstanding  of  the  capabilities  of  terminals  or 
graphics   devices,  neglecting  the  size  of  lines  on  a 
lineprinter  or  the  interruot  capabilities   of   peri- 
pheral oev  ices. 
Db  :   Data  Design  Error 

The  data  design  does  not  fit  trie  needs  of  the  pro- 
gram. Typical  examples  are  wrona  sizes  of  arrays, 
buffers  or  freelists.  Also  cases  of  inappropriate 
data  definitions,  such  that  the  chosen  data  types  do 
not  accomplish  the  desired  precision.  This  error 
tyoe  also  includes  design  of  inappropriate  I/O  for- 
mats and  imorooer  design  of  common  data. 
D /  :   Initialzation  Error 

wrong  or  incomplete  initialization  such  as  forgotten 
initialization  of  global  pointers,  arrays  or  flag 
variables  or  forgotten  initialization  of  interrupt 
hang  1 ers  . 

Db  :   Inadequate  Checkina 

Checking  of  variaoles  or  input  data  is  incomplete  or 
wrong  according  to  program  specifications. 
DV  :   Extreme  Conditions  Neglected 

Data  or  machine  oeoendent  extreme  conditions  are 
neglected,  such  as  numeric  values  which  cause  un- 
derflow, overflow,  excess  of  array  limits  or  maximum 
values  of  real  numbers  or  integers,  ana  occurrence 
of  zero  or  negative  values  due  to  round  off. 
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D  1  0 :   Sequencing  Error 

Program   events   or  decisions  scheduled  in  wrong  se- 
quence such  as  transformation  from  problem  statement 
into  a  sequence  of  operations. 
Dll:   Indexing  Error 

Faulty   index   calculation   such  as  designing  an  er- 
roneous algorithm  for  accessing   array   elements   or 
items  of  a  table. 
012:   Looo  Control  Errors 

Error  in  controlling  either  an  iterative  or  condi- 
tional reoetitive  algorithm,  such  as  basing  exit  or 
break  decisions  on  conditions  which  can  never  occur. 
013:   Misuse  of  Boolean  ExDression 

An  error  is  made  in  usino  Boolean  operators,  such  as 
constructing  a  false  inverse  of  an  expression.  A 
trivial  examnle  of  this  error  type  is  solving 

(  x  +  y  )  '  *  z  =  (  x  '  +  y ' ]  *     z 
instead  of: 

(x+y)'*z=(x'  *  y  '  )  *  Z 
where  +  ,  * ,     '     are  the  boolean  operators  "OR",  "AND", 
and  "NOT"  respectively. 
01  '4  :   Mathematical  Error 

The  mathematical  solution  of  a  problem  within  the 
programming  oroject  does  not  or  not  always  represent 
the  correct  result  according  to  the  program  specifi- 
cations.    This    includes    cases    in    which    a 
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mathematical  equation  is  simply  wrong/  like  a  wrong 
formula  for  calculating  a  mean  or  variance  of  given 
set  of  numbers  and  cases  in  which  the  mathematical 
solution  does  not  apply  for  all  possible  inputs* 
such  as  solving  a  square  root  for  negative  expres- 
sions. 

D15:   Representation  Error 

An  error  is  made  in  the  process  of  the  physical 
representation  of  thoughts/  such  as  writing  design 
documentation  different  from  what  the  designer  had 
i  n tended . 

D16:   M i sunder s t andi ng  of  Problem  Specifications 
Error  resulting  from   misunderstanding   the   problem 
specifications.     For   example   a   decision   table/ 
develooed  during   design/   includes   some   undesired 
combinations  of  actions. 
017:   Other  Design  Errors 

Any  other  design  error  which  is  not  one  of  the  types 
1  istea  above . 


2 .   Coding  Errors 

CI  :   m i sunde r s t ana i ng  of  Design 

An  error  made  in  interpreting  the  desicn   specifica- 
tions/  such   as  misunderstanqing  an  algorithm/  data 


-  24  - 


descriDtions  or  critical  terms*  such  as  mathematical 
expressions   which   leads  to  an  error  in  implementa- 
tion of  the  underlying  design. 
Z2.     :   Negligence 

Parts  of  the  design  are  neglected  or  necessary 
cooperation  with  the  designer(s)  is  omitted.  This 
will  include  cases  where  programmers  are  uncertain 
about  the  design  or  parts  of  it  and  fail  to  take 
appropriate  action,  such  as  requesting  an  explana- 
tion from  the  designer(s)  or  obtaining  the  missing 
information  from  other  sources  such  as  project 
specifications,  hardware  or  software  manuals. 
C3  :   T/0  Format  Error 

Use  of  improper  I/O  formats  in  coding,  such  as  using 
formatted  I/O  when  nonformatted  I/O  is  specified   or 
using  inappropriate  record  lengths. 
C U     :   Misolaced  Data  Declaration 

Error  made  in  declaring  data  either  in  the  wrong 
block  level  (while  coding  in  a  block  structured 
languaqe)  or  in  an  inappropriate  position  within  the 
sou  rce  code  . 

C5  :   Multiple  Data  Declaration 
Multiple  declarations  of  same  data. 
Co  :   Missinq  Data  Declaration 
Necessary  data  declaration  left  out. 
C7  :   Inadequate  Data 
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Error  made  in  choosing  size  or  tvDe  of  data*  such  as 
declarinq  single  precision  instead  of  double  preci- 
sion. 

C8  :   Initialization  Error 
Wrong  or  forgotten  initialization. 
C9  :   Error  in  Parameter  Passing 

Passing  of  parameters  is  incomplete*  wrong  or  tvpes 
do  not  match. 

CIO:   Inadeauate  or  Forgotten  Checking 

Checking  of  inout  is  either  incomolete  or  wrong 
according  to  program  design  spec i f i c a t i ons ,  such  as 
not  checkina  for  invalia  inout  within  a  subroutine 
which  receives  characters  from  a  user  terminal.  A 
typical  case  of  this  error  type  occurs  when  numeri- 
cal inout  is  expected  and  is  converted  to  binary 
without  checkinq  the  validity  of  the  input  charac- 
ter. Another  case  is  an  inadequate  check  of  the 
size  of  the  input  number. 
Cll:   Level  Problems 

while  coding  in  a  block  structured  language  an  error 
results  from  confusing  block  levels*  such  as 
misunderstanding  the  scooe  of  variables*  e.g.  an 
ALGOL  programmer  tries  to  use  variables  or  data  of 
a  lower  level  within  the  main  level  (level  1)  of  a 
o  rog  r am  . 
C  1  2  :   Missing  Declarations  of  Block  Limits 


-  2b    - 


v^hile  coding  in  a  block  structured  language  mandato- 
ry declarations  of  bloc*  level  limits  such  as  BEGIN, 
END  are  left  cut  . 
C13:   Case  selection  error 

An  error  is  made  in  connection  with  a  CASE  statement 
or  comouted  GO  TO  statement  or  an  equivalent  con- 
struct. For  examole  in  an  imol e^entat  ion  of  a  GO  TO 
SwITCH  in  C  V  S  -  2  an  error  occurs  due  to  wrong  se- 
quence of  labels.  C  M  S  -  2  is  a  proqramming  languaqe 
usea  for  tactical  programming  within  military  appli- 
cations t  361  . 
Cl«:   GO  TO  Problems 

An  error  is  made  in  connection  with  the  use  of  a   GO 
TO  stat  ement  . 
C15:   Comment  Error 

Stating  a  faulty  or  misleading  comment. 
C16:   Forgotten  Delimiter 
Delimiter  left  out. 
C17:   Inconsistency  in  Naming 

An  error  is  made  when  naming  an  identifier  in  dif- 
ferent ways.  For  example  a  subroutine  declared  as 
GETCHAR  is  called  by  using  GETCH. 
C 1 8 :  wrona  Use  of  Nested  IF  Statements 
An  error  is  made  durinq  the  construction  of  a  nested 
IF  statement.  Errors  of  this  type  occur  in 
transforming   a   flowchart   or   decision  table  logic 
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into  nested  IF  statements  or  using  an  incorrect  for- 
mat  which  will  cause  a  compile  error  or  associating 
any  of  the  ELSE  cases  with  a  wrong  IF  clause. 
C  1  9  :        Indexing  Error 

Indexing  is  not  appropriate  or  left  out. 
C20:   Inconsistent  Use  of  Variables  or  Data 
An  error  occurs  because  variables  or  data  are       used 
for   conflicting   purposes.    For  example  usage  of  a 
temoorary  storage  location  in  aifferent  carts  of  the 
program   coulo   oe  implemented  in  such  a  manner  that 
information,  is  overwritten  bv   some   subroutine   but 
still   be   needed   by   another   subroutine.   Another 
example  is  the  usaae  of  common   variables   for   dif- 
ferent ourooses  like  oointer  and  index. 
C21:   Sequencing  Error 

An  improper  seauence  of  statements  which  leads  to  an 
error.  For  example  an  exchange  of  array  elements  is 
o  rog  rammed 

TEMP  «-  A(i) 

A(  j  )  *■  TE^P 

Mi)  «■  A  (  j  ) 
which   does   not   accomplish  the  desired  exchange  of 
the  two  el emen  t  s  . 
C22:   Flag  Usage  Problems 

An  error  is  made  in  using  flags  in  order  to  control 
prog  ram  1 oq i  c . 
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C<?3 :   Syntax  Error 

Violation  of  syntax  rules  of  the  programming 
language.  This  includes  errors  in  connection  with 
transformation  of  boolean  exoressions  and  arithmetic 
expressions. 

C24:        Loop  Control  Error 

Error  in  implementing  the  control  structures   of   an 
iterative  or  conditional  repetitive  alaorithm. 
Cd'S:        Incorrect  Exit  from  Subroutines 

An   error   occurs   in  the  implementation  of  an   exit 
from  a  subroutine. 
C26:   Language  Usane  Problems 

An  error  is  made  due  to  idiosyncrasies  of   the   pro- 
gramming  lancuaae   which  are    not  thoroughly  under- 
stood by  the  programmer. 
C<£7;   Forgotten  Statements 

An  error  is  made  by  leaving  out  a   necessary   state- 
ment which  t^e  programmer  knew  was  necessary. 
C  2  8  :   rteoresentation  Error 

A  cod  i  no  error  which  is  introduced  during  the  pro- 
cess of  the  physical  reoresentation  of  thoughts/ 
such  as  writing  a  statement  different  from  what  one 
i  nt ended . 

C29:   Control  Seauence  Error 

An  error  is  made  which  causes  the  program  to  follow 
a  wrong  branch  after  a  decision. 
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C30:   Incorrect  Subroutine  Usage 

Using   the   wrong   subroutine   or  using  a  subroutine 

i naporoo r i a t e 1 y f     e.g.  a  programmer's  expectation   of 

what   a  subroutine  will  accomplish  is  different  from 

what  the  subroutine  actually  does. 

C31:   Other  Coding  Errors 

Any  other  coding  error  which  is  is  not   one   of   the 

types  listed  above. 


3  .   Clerical  Errors 

Al  :   Manual  Error 

An   error   resulting   from   lack  of  motoric  skill  or 

temporary  manual  misfunction,  such  as 

Errors   of   Commission   (e.g.  writing  or  typing 
"busu"  instead  of  "busy") 

Errors   of   Omission   (e.g.   writing   or  typing 
"BOWN"  instead  of  "BROWN") 

Errors  of  Transposition  (e.g.  writing  or  typing 
"hte"  instead  of  "the") 
A2  :   Ment  a  1  Error 

An  error  resulting  from  a  mental   misfunction,   such 
as 

-  Perceptual  Errors  (e.g.  reading  "0"   instead   of 
"0",  11  instead  of  11  or  "DOG"  instead  of  "D00" ) 
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-  Expectation  Errors  (e.g.  in  a  given  context  a 
certain  word  or  exoression  is  exoected  and  the 
actual  word  or  expression  is  overlooked*  such  as 
assuming  "FOR  I  :  =N  STEP  1  UNTIL  i*1  DO"  where  "FOR 
N:  =  l  STEP  -1  UNTIL  M  DO"  is  the  the  actual  im- 
plementation of  an  iterative  looo  in  ALGOL  W.) 
A3  :   Procedural  Errors 

Given  a  proper  assionment  for  a  clerical  task  which 
implies  some  well  defined  sequence  of  actions  which 
could  be  expected  from  the  assignment*  some  of  the 
steps  are  either  forgotten  or  not  carried  out  prop- 
erly* such  as  some  lines  of  code  on  a  coding  sheet 
are  omitted  by  the  keypunch  operator  or  card  correc- 
tion procedures  are  not  followed  appropriately  or 
inserted  cards  are  misDlaced. 
Aa  :   Other  Clerical  Errors 

Any  other  clerical  error  which  is   not   one   of   the 
tyoes  listed  above. 


4.   Debuaging   Errors 

Bl  :   Inappropriate  Use  of  Deougaing  Tools 
tironq       selection  of  debuggina  tools*  such  as  traces* 
snapshots  or  aumos  or   using   them   inappropriately. 
For  example  using  a  snapshot  before  instead  of  after 
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a  certain  instruction  or   taking   a   dump   of   wrong 

memory  locations. 

B2  :   Insufficient  or  I  nape roo r i a t e  Selection  of 

Test  Cases  or  Test  Data 
The  test  cases  or   test   data   provide   insufficient 
test   coverage   of   the   proaram   or  fail  to  orovide 
test  ina  of  critical  oarts  of  the  orogram. 
B  3  :   N'isinteroretation  of  Debugging  Results 
An   i napo rop r i a t e  action   is   taken   due   to   misin- 
teroretation   of   debugging  results/  such  as  a  wrong 
result   is   overlooked   or   misunderstanding   of    a 
correct  result  leads  to  additional  errors. 
BU  :   Misinterpretation  of  the  Error  Source 
An   i napo roo r i a t e   action  in  correct  ina  a  discovered 
software  error  that  results  from   wrong   assumptions 
about  the  error  source*  such  as  correcting  the  symp- 
toms of  an  error  rather  than  its  cause  or   the   pro- 
grammer  fails   to  foresee  the  i^oact  of  a  change  on 
other  parts  of  the  program   and   inserts   additional 
errors  into  the  orogram. 
65  :   Neoligence 

Errors   resultina  from  negligence  in  aebugging.   For 
examole  the  implementation  of  an  error  correction  is 
assumed  to  be  error  free  without  further  checking. 
Bo  :   Other  Debuaging  Errors 
Any   other   debuggina   error  which  is  not  one  of  the 
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types  listed  above 


5.   Test  ina  Errors 


Tl  :   Inaaeauate  Test  Case(s)  or  Test  Data 
Selected   test   case(s)  or  test  data  do  not  prooerly 
test  the  d  rogram  . 

T  <£  :   Mi  s  i  nt  eroret  at  i  on  of  Test  Results 
An  inaoprooriate  action  is  taken   due   to   misunder- 
standing  of  test  results*  such  as  a  wrong  result  is 
overlooked  or  misunderstanding  of  a   correct   result 
leads  to  an    i to  1 emen t a t i on  of  additional  errors. 
T3  :   -Misinterpretation  of  Problem  Specifications 
Error(s)  resulting  from  misunderstanding  the  program 
SDeci  f ications,  such  as  incorrect  interpretation   of 
user   defined  functions   which  leads  to  an  insertion 
of  an  error  or  to  over  looking  an  existing  error. 
T U     :   Meg  1 i  gence 

Errors  result ina  from  negligence  in  testing.  For 
examole  a  tactical  real  time  system  is  merely  tested 
within  specified  limits  and  tests  fail  to  examine 
the  behavior  of  t^e  system  for  cases  where  limits 
are  exceded.  In  this  oarticular  example  negligence 
can  cause  serious  results  in  critical  situations. 
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T5  :   Other  Testing  Errors 

Any   other   testing   error   which   is  not  one  of  the 

tyoes  listed  above. 


In  order  to  get  meaningful  results*  the  above  defini- 
tions were  used  throughout  the  project.  This  list  is  not 
considered  complete.  However/  it  permitted  identification 
of  software  error  types  for  the  puroose  of  this  exoeriment. 
Some  of  the  listed  error  tvoes  could  be  further  subdivided 
into  subsets  of  eacn   type  if  this  were  desired. 

To  contribute  to  analysis  of  error  sources/  it  was 
decided  to  list  the  same  error  type  which  occurs  in  dif- 
ferent software  development  phases  under  different  ca- 
tegories. For  examole  one  type  of  error  which  has  the  same 
name  (loon  control  error)  may  occur  in  both  design  and  cod- 
ing phases.  However/  the  cause  of  the  error  could  be  dif- 
ferent . 

Error  severity  is  related  to  the  effect  of  the  error  on 
system  operation.  There  is  no  direct  relationship  between 
error  tyoe  and  severity.  For  examole  mistyping  of  a  single 
character  of  a  variable  name  in  FORTRAN  can  cause  strange 
results/  whereas  the  same  error  made  in  a  comment  mav  not 
affect  the  program  at  all.  The  consideration  of  error 
severity  was  beyond  the  scope  of  this  Droject. 
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IV.   EXPERIMENT  DESCRIPTION 


A.   GENERAL  REMARKS 

Error  data  were  gathered  in  four  programming  projects 
ranginq  from  small  to  large.  During  280  man  hours  of  total 
project  time  more  than  2000  source  statements  were  produced. 
A  total  of  173  errors  were  recorded. 

The  scooe  of  the  programming  Drojects  was  limited  to  the 
following  software  development  phases: 

-  Design  and  Design  Review 

-  Coding 

-  Debugging 

-  Testing 

6.   DESIGN  OF  THE  EXPERIMENT 

The  experiment  was  designed  to  guarantee  the  most  accu- 
rate recording  of  relevant  error  information.  Therefore/ 
for  each  of  these  ohases*  an  appropriate  form  was  designed 
to  give  a  firm  ouiceline  to  the  experiment  programmer  for 
recordina  the  entire  development  of  a  orogram  including 
necessary  details  about  errors.  The  error  data  which  were 
recordeo  on  these  forms  for  project  1/  2.  t  3  and  4  are  given 
in   Aopendices  A,  9,  C  and  D  respectively.   For  each  project 
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the  recorded  data  was  analyzed  with  respect  to  the  following 
factors: 

-  Number  of  source  statements  oroduced 

-  Van    hours  scent  on  the  project 

-  Man  hours  scent  in  each  of  the  software 
develooment  phases 

-  CPU  time  used  for  compiles 

-  CPU  time  used  for  test  and  debug  runs 

-  Number  of  test  and  debugging  steos  needed 

-  Project  time  used  to  correct  errors 

-  Project  time  between  error  detections 

-  Occurrence  cf  errors  with  resoect  to  software 
develooment  ohases 

-  Detection  of  errors  with  resoect  to  software 
develooment  ohases 

-  Time  history  of  error  detection  and  correction 

-  Complexity  measures  of  suoroutines 

4  summary  of  this  analysis  is  documented  at  the  end  of 
eac  h  append  i  x . 

Since  it  would  be  too  difficult  to  obtain  error  data  in 
a  commercial  environments  it  was  neccessary  to  obtain  the 
data  by  usinq  a  careful  1 v  controlled  experiment  in  an 
acaoemic  environment.  The  considerable  overhead  necessary 
to  record  all  relevant  data  was  not  included  in  project 
time.  It  orobaoly  woulo  not  be  feasible  to  conduct  this 
kind  of  exoeriment  in  a  commercial  environment  because: 
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-  The  overhead  would  be  very  costly.   It  is  unlikely  that 
management  would  supoort  an  experiment  of  this  type. 

-  Competition  among  programmers  would  probably  not  allow 
accurate  results  to  be  obtained. 

-  Programmers  may  feel  that  the  error  data  would  be 
used  for  performance  evaluation. 

In  addition  to  qathering  quantitative  error  data  an 
attempt  was  made  to  obtain  other  information  about  the  na- 
ture of  errors  such  as  "why  was  the  error  made?"  or  "how  was 
the  error  discovered?".  This  information  could  help  to  dev- 
ise methods  for  avoioinn  certain  types  of  errors.  For  each 
subroutine  a  directed  graph  representation  was  analyzed  with 
respect  to  the  following  complexity  measures: 

-  N umber  of  Nodes 

-  Numbe  r  of  Arcs 

-  Number  of  Statements 

-  Numbe r  of  Paths 

-  Reachability  of  Nodes 

-  Cyclomatic    Number     [3<?1 

The  results  of  this  analysis  are  shown  together  with  the 
directed  graph  representation  of  the  programs  in  the  appen- 
dices. For  some  structures  these  measures  are  not  shown 
because  the  number  of  Daths  and  the  reachability  index  was 
too  large  to  calculate. 
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C.   PROGRAMMING  ENVIRONMENT 

The  underlyinq  program  specifications  of  all  projects 
were  well  defined  and  not  known  by  the  experiment  programmer 
before  the  experiment.  The  experiment  programmer  was  fami- 
liar with  the  objectives  of  the  experiment  and  willing  to 
record  accurate  data  to  the  oest  of  his  knowledge,  even  if 
recorded  facts  seemed  to  be  unfavorable  for  himself. 

Throughout  trie  experiment  software  development  was  done 
in  a  structured  and  ohase  oriented  aoDroach  using  structured 
programming  and  known  software  development  techniques^  such 
as  too-down  design,  modularization,  decision  tables,  etc.. 

As  a  oroaramminq  environment  the  OS/360  system  was 
chosen  as  one  of  the  most  representative  batch  processing 
systems  available.  Batch  orocessing  was  always  used  except 
for  the  final  test  ana  debug  runs  for  project  #  4,  because 
this  program  was  desianed  to  run  in  a  time  sharing  environ- 
ment. Durino.  the  exoeriment  all  environmental  factors,  such 
as  ooerating  system  and  eauioment  oeing  used  and  the  pro- 
gramming language  remained  unchanged.  Factors  related  to 
the  experiment  programmer,  such  as  ability,  knowledge,  and 
experience  did  not  vary  considerably  durino  the  experiment. 
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V.   QUANTITATIVE  RESULTS 


A.   DISTRIBUTION  OF  PROGRAMMING  EFFORT 

In   Table   1   the   distribution  of  project  time  is  shown 
with  respect  to  the  major  software  development  phases. 


PROJECT 
PHASE 


PROGRAMMING  EFFORT  (MAN  HOURS) 


Project 
*     1 


Project 
a    2 


Project 
U    3 


pro j  ec  t 

9    a 


Total 


DESIGN 


5.0 
(22.9%) 


31  .0 
(24.«%) 


7.0 
(21  .27.) 


2a. o 
(23.8%) 


67.0 
(23.9%) 


CODING 


7.0 
(32.1%) 


26.0 
(20.8%) 


4.0 
(12.1%) 


2a. 5 
(2a.  3%) 


61  .5 
(21.9%) 


DEBUGGING 


a.o 

(18.3%) 


55.0 
( a  a  .  o  % ) 


3.0 
(  9.1%) 


ai  .5 
(  a  l  .  l  % ) 


103.5 
(36.9%) 


TESTING 


5.8 
(26.6%) 


13.0 
(10.4%) 


19.0 
(57.6%) 


1  1  .0 
(10.9%) 


48.8 
(17.4%) 


SUM 


21  .8 


125.0 


33.0 


101.0 


280.8 


TABLE  1 
DISTRIBUTION  OF  PROJECT  TIME  DURING  THE  EXPERIMENT 

B.   ORIGIN  OF  ERPORS  VS  DETECTION  OF  ERRORS 

The   final   statistics   of   each  project  contains  tables 
showing  the  numbers  of  errors  ^ade    and  the  number  of   errors 
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found   with   respect   to   each   of   the  software  development 
phases.   A  summary  of  these  tables  is  presented  in  Table  2. 


!  #   OF   ERRORS    FOUND 

PROJECT   ! Pro j ect ! Pro j ect  ! Pro j ect ! Pro j ect  !  TOTAL  (Percentage) 
PHASE     i   *  1   !  U    2   !   *  3   !  tt    a       \ 

DESIGN    !     2   !     1   !        !        !    3   (1.7%) 

CODING    J    22   !    21   !     1   !     a   J   48   (27.7  X) 
DEBUGGING!    19  \          53   J     3   J    a5   |  120   (o<?.4  %) 
TESTING   !     1   !        !        !     1   !    2   (  1 . 2  %) 
SUM       ;    44   !    7  5   !     4   i    50   j  173   (100%) 

!   U       OF    FRRORS   MADE 

PROJECT   [Pro ject IProject | Pro ject  !  Pro j ect  !  TOTAL  (Percentage) 
PHASE     !  U    1   ;   s2   !   #3   !   #4   ! 

DESIGN    ;     5   I    2  0   !        j    10   j   35   (20.2  %) 

CODING    !    38   !    53   !     4   J    37   j  132   (7b. 3  %) 

DEBUGGING!     1!     2!        !     3   J    6(3.5%) 

TESTING   !        !        !        J        }    0   (  0.0  %) 

SUM       ;    44   J    75   !     4   •    50   J  1 73   (100%) 

TABLE  2 
ERROR  DETECTION  VS  ERROR  ORIGIN 
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C.  ERROR  CORRECTION  TIME  AND  TIME  BETWEEN  ERRORS 

1.  Error  Correction  Tine 

The   mean   ti^e   to  correct  an  error  calculated  over 
all  173  errors  was  \2.2    man  minutes. 

2 .  Time  Between  Error  Detections 

The  mean  time  between   error   detections   calculated 
over  all  projects  was  61. o  man  minutes. 

D.  ERROR  TYPES 

All  errors  which  occurred  during  the  exoeriment  could  be 
identified  by  one  of  the  Dreviously  defined  error  types.  An 
overview  of  error  tvoes  with  resoect  to  their  frequency  of 
occurrence  is  oresented  in  Table  3.  Error  tyoes  which  did 
not  occur  are  not  listed.  In  addition  to  the  information 
given  in  Table  3  a  bar  chart  (Table  4)  is  shown  which  iden- 
tifies the  most  common  error  tyces. 
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ERROR 
TYPE 

1                        NUMBER    OF    ERRORS                     1T0TAL 
.Project [Project {Project IProject  1 
,       #1        1       #2       I       #3       I       #4       I 

D3 

D7 

D9 

010 

Dl  1 

D12 

013 

D15 

3       1             3       1                     12       18 
1                     I                     1111 

1        1           10        I                      1                      ill 
111                      1                      11 

1       1            3       1                    1                    14 
12       1                    15       17 
1                    1                    1111 
1            1       1                    1112 

CI 

C4 

C5 

Cb 

C7 

C8 

C<? 

CIO 

CI  1 

C12 

C15 

C16 

C17 

C  1  9 

C20 

C21 

C23 

C24 

C2b 

C27 

C28 

C29 

C30 

1       1             1       I                    1                    12 
1       I                    1                    1                    11 
1       1                    1                    1                    11 

1  1                    1112       14 
1                    1                    1111 

2  1            1       1                    !                    13 
1            2       1                    1113 

112       1                    1                    13 

3  12       1                    1                    15 

4  12       1                    1117 
11                    I                    I                    11 

111                    1                    !             1 
115       1                     13       19 
11                     111                     12 
2       111                     1                     13 

1            2       1            1        I            2       1            5 
b       1            5       I                     1             1        1          12 
2       1             1        I             1        I                     14 

III                     1                     11 
2       I             3       1                     1             4       1             9 

5  1             b       I                     I             b       I          17 
I             2       I                     1                     12 
I             1        I                     I                     11 

Al 
A2 
A3 

2       I          14       I                     I          lb       1          32 

2       1            1        1                    1114 

1                    I                    1111 

B3 
B4 

111                    I                    1                    11 
1                    12       1                    1113 

TABLE  3 
ERROR  DISCOVERY  rtlTH  RESPECT  TO  ERROR  TYDES 
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ERROR  ! 

TYPE   !   FREQUENCY  OF  OCCURRENCE 


D3 

09 

01  1 

012 

C6 

CI  1 

C12 

C17 

C21 

C23 

C24 

C27 

C28 

Al 

A2 

Other 

Errors 


******** 
*********** 

*  *  *  * 
******* 

*  *  *  * 

*  **  *  * 
******* 
********* 

*  *  *  *  * 
************ 

*  *  *  * 

********* 
***************** 
******************************** 

*  *  *  * 

*********************************** 


TABLE  4 
MOST  FREQUENT  ERROR  TYPES 

E.   ERROR  OCCURRENCE  RELATED  TO  COMPLEXITY  MEASURES 

One  of  the  objectives  of  the  experiment  was  to  determine 
the  re  1  a t i ons h i p ,  if  any,  between  error  occurrence  and  the 
structural  properties  of  subroutines.  Of  particular  in- 
terest was  the  analysis  of  structural  properties  with 
resoect  to  the  error  simulation  work  done  by  Schneidewind* 
Howard  and  Ki  rchgaessner  [ 3  3 1  •  In  addition  the  cyclomatic 
number  as  defined  by  ivcCabe  132]  for  software  engineering 
purposes/  was  used  in  the  analysis.  The  results  of  this 
analysis  are  shown  in  Table  5.    The   followinq   definitions 


-  U3  - 


and  terms  are    used  in  Table  5: 

N  p    Number  of  Paths  (minimum  number  of  paths:   no 
looo  traversed  more  than  once  in  succession) 

V     Cyclomatic  Number:   number  of  independent  circuits 
number  of  arcs  -  number  of  nodes  +  2 

R     Reachability:   summation,  over  the  nodes,  of  number 
of  ways  of  reaching  a  node 

r     Average  reachability:   P/number  of  nodes 

S     Number  of  Source  Statements 

e     Number  of  Errors  Found  in  Actual  Proqram 

Tf    Labor  Time  Required  to  Find  Errors 
(Since  Previous  Error  Hetection) 

Tc    Labor  Time  Reauired  to  Correct  Error 


All  of  the  above  are  with  respect  to  a  single  program. 
The  definition  of  V  includes  an  implicit  arc  connect inq  the 
start  and  terminal  noces  (strongly  connected  graph). 
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TABLE  5 

COMPLEXITY  MEASURES  VS.  ERROR  PROPERTIES 

Part  A  -  Procedures  With  One  or  More  Errors 

COMPLEXITY  MEASURES  ERROR  PROPERTIES 


Pro  j  ec  t  / 

Procedure    No    V 


Z^Tf     Etc 
(Man-    (Man- 
e    mins)    mins) 


1/1 

2 

2 

7 

l  .a 

14 

1 

35 

10 

1/5 

* 

b 

* 

* 

Bb 

5 

95 

53 

1/b 

* 

5 

* 

* 

7 

2 

37 

35 

1/8 

* 

5 

* 

* 

21 

1 

35 

15 

1/9 

2 

2 

8 

1  .333 

b 

1 

1  15 

20 

2/1. IP 

1 

1 

2 

1  .0 

3 

1 

10 

10 

2/1  .23 

1 

1 

2 

1  .0 

I  1 

1 

1  10 

10 

2/2.2 

2 

1 

a 

1  Jo 

8 

1 

15 

10 

2/7 

3 

2 

7 

1  .a 

15 

3 

230 

45 

2/9 

72 

8 

370 

19.474 

45 

3 

140 

185 

2/10 

9 

a 

25 

2.778 

18 

1 

10 

5 

2/1  1 

* 

b 

* 

* 

54 

3 

950 

b5 

2/12 

5 

2 

13 

l .  a  a  a 

34 

2 

300 

3  0 

2/15 

* 

a 

* 

* 

19 

1 

5 

1 

2/lo 

* 

5 

* 

* 

30 

2 

150 

20 

2/ie 

12 

a 

50 

2.941 

2b 

1 

5 

15 

2/21 

* 

lb 

* 

* 

94 

8 

75  0 

145 

3/3 

2 

2 

b 

1  .5 

13 

1 

80 

5 

*  Very  la rge  vaiue 
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TABLE  5 
(cent  i  nued ) 

Part  A  -  Procedures  With  One  on  More  Errors 


Z>f 

ETc 

Pro  j  ec 

t/ 

(Man- 

(Man- 

Procedure 

Md 

V 

R 

r 

S 

e 

mi  ns  ) 

mi  n  s  ) 

a/7 

ao 

6 

151 

3.a38 

83 

1 

120 

5 

a/1  5 

16 

5 

6a 

a. 267 

28 

1 

2  0 

15 

a/  1  a 

* 

8 

* 

* 

37 

5 

255 

b5 

a/15 

a 

3 

12 

2.0 

13 

2 

ao 

35 

a/21 . 

3 

7 

3 

3a 

a. 857 

16 

1 

0 

5 

a/22 

* 

7 

* 

* 

3a 

1 

lbO 

30 

a/23 

18 

5 

60 

a. 615 

2a 

1 

125 

10 

a/27 

5 

a 

23 

2.091 

18 

3 

360 

120 

a/28 

* 

5 

* 

* 

35 

2 

90 

50 

a/29 

321 

13 

ia68 

5a. 370 

a9 

5 

1  125 

lbO 

a/30 

b 

a 

2a 

2. a 

19 

1 

bO 

30 

a/31 

* 

a 

* 

* 

27 

1 

3  0 

10 

a/33 

ia 

a 

76 

7.o 

17 

2 

135 

10 

*  Very  1 arqe  va 1 ue 
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TABLE  5 
(continued) 


Part  B  -  ComD 

1  ex  i  t  v  Measures  f 

or  P  r oc  edu 

with  Zero 

Errors 

Project/ 

Procedure 

Md    V 

R 

r 

s 

1/2 

3     2 

8 

1  .333 

6 

1/3.1 

2 

1  .0 

8 

1/3.2 

2 

1  .0 

1 1 

1/3.3 

2 

1  .0 

a 

1/4 

o     3 

26 

a. 333 

18 

1/7 

7     3 

ao 

5.0 

15 

2/1.1 

2 

1  .0 

3 

2/1.2 

2 

1  .0 

3 

2/1.3 

2 

1  .0 

3 

2/1  .a 

2 

1  .0 

3 

2/1  .5 

2 

1  .0 

3 

2/1  .6 

2 

1  .0 

3 

2/1  .7 

2 

1  .0 

3 

2/1.8 

2 

1  .0 

3 

2/1  .9 

2 

1  .0 

5 

2/1.10 

2 

1  .0 

5 

2/1.11 

2 

1.0 

5 

2/1.12 

2 

1  .0 

13 

2/1.13 

2 

1  .0 

3 

2/1  .  ia 

2 

1  .0 

3 

2/1.15 

2 

1  .0 

3 

2/1.16 

2 

1  .0 

3 

2/1.17 

2 

1  .0 

3 

2/1.18 

2 

1  .0 

3 

2/1  .20 

2 

1  .0 

3 

a/1.1 

2 

1  .0 

2 

a/l  .2 

2 

1  .0 

2 

a/l  .3 

2 

1  .0 

7 

a/  l  .a 

2 

1  .0 

5 

a/i.5 

2 

1  .0 

7 

a/i  .6 

2 

1  .0 

5 

a/1.7 

2 

1  .0 

5 

a/i  .8 

2 

1  .0 

5 

a/i  .9 

2 

1  .0 

5 

a/i.io 

2 

1  .0 

a 

a/i.n 

2 

1  .0 

3 

a/1.12 

2 

1  .0 

3 

a/i.13 

2 

1  .0 

3 

a/  l .  ia 

2 

1  .0 

3 

a/i .  15 

2 

1  .0 

3 

a/1.16 

2 

1  .0 

3 

a/i.17 

2 

1  .0 

3 
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TABLE  5 
(con t  i  nued ) 

Part  B  -  Comolexity  Measures  for  Procedures 
with  Zero  Errors 


Project/ 

Procedure 

No 

v 

R 

r 

S 

a/1.18 

1 

2 

1  .0 

3 

a/i.i9 

1 

2 

1  .0 

5 

a/i .20 

1 

2 

1  .0 

5 

a/i  .21 

1 

2 

1  .0 

6 

a/i  ,22 

1 

2 

1  .0 

9 

a/i .23 

1 

2 

1  .0 

6 

a/2,  l 

2 

a 

1  .0 

8 

a/2.2 

2 

a 

1  .0 

9 

a/2.3 

2 

a 

1  .0 

9 

a/3 

2 

2 

6 

1.5 

a 

a/a.  i 

2 

2 

8 

1  .6 

7 

a/a. 2 

2 

2 

8 

1  .6 

Q 

a/5 

8 

a 

38 

a. 222 

56 

a/6 

a 

1 

6 

1  .0 

2a 

a/8.1 

2 

2 

8 

1  .6 

13 

a/8.2 

2 

2 

8 

1  .6 

13 

a/8.3 

2 

2 

8 

1  .6 

10 

a/8. a 

2 

2 

8 

1  .6 

9 

a/8.5 

2 

2 

8 

1  .6 

12 

a/9 

16 

5 

95 

7.917 

21 

a/io 

12 

5 

65 

a.6a3 

a9 

a/i  l 

7 

3 

38 

5.a29 

19 

a/12 

* 

a 

* 

* 

20 

a/16. l 

2 

2 

6 

1.5 

6 

a/16.2 

2 

2 

6 

1  .5 

12 

a/16.3 

2 

2 

6 

1  .5 

9 

a/16. a 

2 

2 

6 

1.5 

10 

a/17 

16 

1 

18 

1.0 

21 

a/18 

8 

a 

a2 

3.818 

21 

a/19 

a 

3 

16 

2.667 

1  1 

a/20 

3 

2 

9 

1  .  125 

13 

a  /  2 1 .  l 

7 

3 

3a 

a. 857 

ta 

a/2a 

lb 

7 

83 

a. 368 

19 

a/25.i 

2 

2 

8 

1  .333 

15 

a/25.2 

2 

2 

8 

1  .333 

10 

a/25.3 

2 

2 

8 

1  .333 

17 

a/26 

3 

3 

15 

1  .5 

19 

a/32 

7 

3 

36 

5.  ia3 

15 

a/3a 

2 

2 

5 

1  .25 

15 

*  Very  larce  value 
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Using   the   results   of  Part  A  in  Table  5 ,  the  following 
correlation  coefficients  were  calculated: 


v  vs. 

S  vs. 

V  vs. 

V  vs. 
S  vs . 
S  vs . 

V  vs. 


e   : 

e  : 
ZTf 
ZTc 

ZTf 
2Tc 

S   : 


.783a 
.5880 

.6734 

.7229 

.5902 
.5091 
.7903 


The  relationship   between   mean   complexity   values   and 
error  occurrence  is  shown  in  Table  6. 


Mean  Cvclomatic 
Nu^be  r  ( V ) 


Zero  Errors 
8 1  P  roceau  res 
1  .68 


r4ean  N  u  m  o  e  r  of  Source 
Stat  emen t  s  ( S ) 


9.33 


One  Or  More  Errors 
3  1  Procedures 
a.  7a 

27.23 


TABLE  o 
RELATIONSHIP  BETWEEN  MEAN  COMPLEXITY 
VALUES  A^IO  ERRORS  FOUND 
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VI.   SUMMARY  AND  CONCLUSIONS 


A.  GENERAL  REMARKS 

Due  to  limited  man  power  and  time  constraints  the  amount 
of  test  data  was  too  small  to  allow  major  conclusions. 
Furthermore  the  scope  of  the  experiment  did  not  include  some 
important  programming  problems  such  as  real  time  and 
mathematical  problems.  However,  taking  all  constraints  into 
consideration  some  Qualitative  aspects  were  recognized  which 
could  be  important  factors  in  subsequent  studies. 

B.  SIGNIFICANCE  OF  ERROR  TYPES 

Although  the  categorization  and  definition  of  error 
tyoes  was  not  considered  complete;  all  errors  detected  in 
the  experiment  could  be  identified  using  these  error  defini- 
tions. The  most  freouent  error  type  "Al"  (Manual  Error) 
seems  to  be  relatea  to  the  number  of  source  statements 
whereas  the  frequencies  of  the  remaining  errors  seem  to  oe 
related  to  the  complexity  of  the  programming  problems. 

The  recording  of  errors  with  resoect  to  their  types  con- 
tributes to  a  learning  process  which  enables  the  programmer 
to  reduce  the  number  of  errors  made  on  subsequent  projects. 
It  was  felt  that  this  concept  could  also  be  effectively  used 
in  a  commercial  software  production  environment.  The  error 
recording   would   have   to   be   private   to   the   individual 
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programmer  and  used  by  the  proarammer  alone.  Otherwise  it 
would  have  a  negative  influence  on  working  atmosphere  and 
performance.  If  the  evaluation  process  were  assisted  by 
appropriate  software  packages,  the  overhead  in  analyzing 
errors  would  be  reduced  to  a  great  extent.  The  overhead  of 
recording  the  errors  for  the  individual  programmer  tends  to 
decrease  after  the  recording  system  has  been  learned. 

C.   COMPLEXITY  MEASURES 

Several  measures  of  complexity  calculated  for  each 
subroutine  are  presented  in  the  aopendencis  together  with 
subroutine  directed  graph  reoresentation.  when  the  number 
of  errors  found  in  procedures  was  correlated  with  cyclomat- 
ic  number  and  number  o f  source  statements?  the  correlation 
coefficients  were  hioher  than  for  other  complexity  measures 
[  3  4  J  .  It  also  aDoeared  that  these  two  measures  were  related 
to  the  total  error   detection   and   total   error   correction 

— 7 

times.  It  was  learned  that  trying  to  keeo  the  cyclomatic 
number  small  not  only  reduced  the  numcer  of  errors  but  also 
contributed  to  the  reduction  of  oebugging  and  testing  ef- 
fort. This  experience  suoports  the  results  of  M  c  C  a  b  e  '  s 
analysis  of  structural  properties  of  programs  with  respect 
to  difficulties  in  testing  and  debugging  1321.  It  should  be 
noted  that  the  influence  of  other  factors  such  as  usage  of 
structurea  programming  techniques  and  working  habits  of  the 
experiment  programmer  also  contributed  to  these  results. 
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D.   SOFTWARE  ENGINEERING  ASPECTS 

Throughout  the  experiment  comments  were  made  to  indicate 
possiole  clues  about  error  sources  or  ways  to  avoid  certain 
types  of  errors.  An  evaluation  of  these  comments  showed 
that  some  of  these  errors  would  have  been  avoided  by  the 
usage  of  decision  tables  or  aoDlying  a  prooer  desk  test. 
Many  key  punch  errors  of  tyoe  "M"  were  not  recognized  dur- 
ing the  orocess  of  ounchina  caros»  because  the  design  of  the 
key  punch  machines  (IBM  2  9 )  does  net  proviae  an  immediate 
control  of  characters  being  punched.  It  is  aesirable  to  see 
the  result  of  each  key  stroke  instantaneously.  The  freauen- 
cy  of  error  occurrence  of  type  "Manual  Error"  in  former  pro- 
jects done  by  the  same  programmer  using  console  editing  was 
considerably  lower  than  the  error  frequency  discovered  in 
this  experiment. 

Software  engineering  concents  such  as  top-down  design, 
design  review  and  Structured  walk  Through  were  used 
throughout  all  projects.  These  methoas  had  a  major  influ- 
ence on  the  software  oevelopment  process  and  were  considered 
to  be  extremely  useful. 

The  usage  of  ALGOL  W  had  a  major  influence  on  the  pro- 
gramming style.  Variable  names  were  chosen  to  make  programs 
self-documenting  to  a  great  extent.  Therefore  most  program 
parts  were  easy  to  read  and  to  understand  which  implied  a 
reduction  of  the  debugqing  effort. 
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Another  concept  which  was  successfully  employed  was  the 
idea  of  using  identifiers  which  differ  by  two  or  more  char- 
acters. The  idea  of  implementing  some  kind  of  "Hammina  Dis- 
tance" (351  between  identifiers  was  derived  from  bitter 
experience  in  former  projects  and  helDed  to  avoid  errors 
which  occur  when  a  single  error  of  tyoe  "Manual  Error"  or 
"Mental  Error"  produces  a  different  identifier  which  matches 
with  one  of  the  previously  aefined  identifiers.  Thus  this 
kind  of  error  did  not  occur  during  the  compiling  process. 
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E.   CONCLUSIONS 

Errors  in  software  orojects  aooear  to  be  related  to  pro- 
gram structure  and  tc  the  number  of  source  statements. 
Althouah  this  experiment  was  not  larqe  enough  to  state  these 
relations  in  a  formal  manner,  the  results  were  encouraging. 
Cyclomatic  numoer  and  number  of  source  statements  could 
prove  useful  as  guidelines  for  managinq  proaramming  pre 
j  ect  s . 

The  usage  of  too-down  desian,  structured  programming 
techniques  and  other  software  development  tools  should  be 
mandatory  for  orogramminq  projects.  The  recording  and 
evaluation  of  software  errors  would  be  a  reasonable  accroach 
to  avoid  errors  in  programming  orojects  and  to  reduce  cost. 

Detailed  debugging  on  a  modular  basis  and  development  of 
large  orograms  by  stecwise  integration  of  thoroughly  de- 
bugged  modules   seems   to   be   a   good   strategy  to  achieve 
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reliable  programs. 

F.   RECOMMENDATIONS 

Similar  experiments  would  reveal  more  of  the  relations 
between  software  errors  and  complexity  of  programs.  More 
definite  information  about  the  utility  of  complexity  meas- 
ures coula  be  obtainec  by  using  these  measures  on  large- 
scale  software  projects  in  a  commercial  production  environ- 
ment . 
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APPENDIX  A 
PROJECT  DESCRIPTION 
Project  n     :  1 

Project  title  :  PALINDROMES 
Programmer  :  HOFFMANN 
Programming  Language  :  ALGOL 

Programming  environment:  IBM/360/67 , OS/MVT , BATCH 
Design  notes  :  see  ANNEX  A 
Program  listing  :  see  ANNEX  B 
Coding  notes  :   see  ANNEX  C 
Debugging  notes  :   see  ANNEX  D 
Error  Listing  :    see  ANNEX  E 
Final  statistics  :   see  ANNEX  F 
Graphical  representation  :  see  ANNEX  G 
Test  phase  description:  see  ANNEX  H 


Starting  date:  28  JAN  77 


Enai  ng  date:  6  FEB  77 


EXPERIMENT  DESCRIPTION 


1  .  Project  description: 

FIND  PALINDROMS 
A  palindrome  is  defined  as  a  character  string  of 
n,  which  has  the  followino.  characteristics: 


length 


character  (il  =  charactertn  -  i  -  11  for  all  i  from 
i=0  to   i  =  n  /  2  (inteaer  division)  and  n  >=  2 
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Given   an  input  string  of  length  n*  where  2  <=  n  <=  256, 
find  all  occurrences  of  palindromes. 
The  input  is  given  in  form  of  punched  cards. 

The  length  of  the  string  ("1")  preceeas  the  input  of  the 
Character  string. 

All  information  being  read  should  be  printed. 

Give  as  a  result  the  number  of  palindromes  of  each  length 
and  the  position  of  the  starting  character  of  each  palin- 
drome in  card  number  and  character  dosi t ion. 

Find  only  maximal  length  palindromes,  (i.e.  "ABC  C8A"  should 
be  recorded  as  the  palindrome  of  maximal  length/  whereas  the 
included  palindromes  "BC  CB"  and  "C  C"  should  be  exclud- 
ed.) 

2.   Programmer's  background: 

a)  Experience  in  programming: 

Oct  1970  -  May   1971    Programming  courses 

May  1971  -  April  1972   Module  Programmer 

May  1972  -  June   197a   work  in  Test  and  Simulation  Depart- 
ment at  the 

NAVAL  COMMAND  AND  CONTROL  SYSTEMS 
COMMAND  (FEDERAL  GERMAN  NAVY) 
Testing  of  tactical  real  time  systems 
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March  1975  -  Jan  1977   Student  at  the  NAVAL  POSTGRADUATE 

SCHOOL/  Monterey*  Computer  Science 

b)  Experience  in  testing: 

Two  years  of  work  in  testing  and  simulation. 

c)  Experience  in  the  area  of  the  qiven  problem:   None. 

d)  Experience  in  the  progamming  language  being  used: 
Experience   over   a   period  of  18  months  in  more  than  10 

orogramming  projects.   (Total  number   of   source   statements 
produced  during  that  time  was  more  4000.) 

3.   Psychological  factors: 


a)  Did  the  programmer  like  the  project?   Yes. 

b)  How  ooes  the  programmer  like  the  orogramming 
1 anguage? 

Favorite  programming  language. 

c)  was  the  programmer  satisfied  by  the  way  the  problem 
was  soec  i  f  i  ed? 

Only  minor  criticism. 

d)  How  did  the  programmer  like  the  Drogramming  environ- 
ment ? 

The  facilities  (study  room/  card  punch   room)   were   not 
conducive   to   efficient   orogramming   because  of  restricted 
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space*  bad  lighting  and  noise. 

e)  Other  factors: 

The  recording  of  the  exoeriment's  data  during   the   pre 
ject  affected  speed  and  concentration  considerably. 
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4 .  Comments  on  Documentation 

For  the  documentation  of  each  software  development  phase  a 
special  documentation  form  has  been  developed.  These  forms 
are  designed  to  provide  a  f  i  r<s\  gu  i  de  1  i  ne  for  the  experiment 
programmer  to  record  all  data  of  interest  for  subsequent 
error  analysis. 

-  Beain  and  end  of  each  step  was  recorded  with  resoect  to 
day  and  t  i  me  . 

-  Each  error  was  recorded  when  it  is  discovered.  The  error 
was  then  identified  by  a  unigue  error  number  (1/2/...). 
Furthermore  the  time  of  discovery  and  the  error  tyoe  (using 
error  types  listed  in  ANNEX  F)  were  recorded. 

-  If  appropriate/  comments  about  error  discovery/  reason  why 
the  error  was  made/  etc.  were  documented  in  ANNEX  E2. 

-  For  each  error  the  phase  in  which  the  error  was  made/  the 
phase  in  which  the  error  was  discovered  and  the  time  scent 
to  correct  the  error  was  recorded  in  ANNEX  El. 

For  each  steo  in  any  one  of  the  software  develooment 
phases  the  day/time  of  begin  and  end  was  recorded.  In  addi- 
tion/ the  time  (in  man  hours)  for  each  step  was  recorded. 
This  excludes  the  overhead  used  for  documentation  of  the 
exper  i  men t  data. 
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ANNEX  A 


Page  1   of  « 


WORKSHEET  FOR  DESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  « 


STEP 

« 


PROBLEM  ANO  PLANNED  SOLUTION 


ALTERNATE 
SOLUTIONS 


DAY 
TIME 


MAN  | 
HOURS.'ERROR 
/STEP!   « 


COMMENT 


Too-Down 
Use    the    f 
bl ock s: 
•Initiali/ 

•Reaa    and 

•F  i  nd  all 
are  not  t 
a  I arger 
rind  also 
*h  i  c h  ar^ 
ABBAXA  mi 
two  o  a  1  i  n 
ABBA  a 

■write  all 
found 

•The  neces 

1 .  1 engt h 

2.  charac 
1 enqt  h 

are  e*oec 


Oes  i  qn : 

ol lowing  buildina 

at  i  on 

write  Inout  Cards 
Pal  indro^es  which 
ot  a  I  I v  i  nc 1 uded  i  n 
oa I i  norome 

oa 1 i  ndr omes 

over  I  aoo i  no. : 
1  I  be  recorded  as 
dromes : 
nd  Ax  A 

palindromes  beina 

Sary  incuts 

of  String 
ter  string  of  given 

t  ed  f  rom  cards . 


Soecify  Initialization: 
•Initialize  all  arrays  and 

variables 
-Write  explanatory  test 
-Read  length  of  inout  stMnq 

and  check(assert  2<  =  lenqth 

Of  Strinq  <=  256) 
-Set  integer  field  size  to  5 
-Use  all  80  columns  of  the 

input  cards  (specify  as  com4 

pile  parameter  CARDLIMIT) 


-Do  not  allow 
over  1 app  »  ng 
na 1 i  ndromes  • 
This  a  1  I OwS  t  wo 
solutions: 
1  .  Always  record 
the  first  pa  1 i  n- 
drome . 

2 .  Always  record 
the  la  rger  oa- 
1  i  ndrome .  If  ca- 
1  i  ndromes  are  of 
eaua 1  1 engt  h , 
then  record  the 
first  one. 


Cardl i  mi  t  ecu  1 d 
be  soec  i  f  i  ed  bv 
user  via  i  ncut 
parameter . 


1/28 

10UO 


1.5 


1/28 

1330 

t/28 

1330 

.5 


1/28 

uoo 


Remarks  J 

Because  of  the  small  size  of  the  project  the  system  design  phase 
is  omi  t  ted. 

Total  man  hours  soent  in  design:  5.0 
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ANNEX  A 


Page  2   o*  t 


WORKSHEET  FOR  DESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  * 


STEP 
U 


PROBLEM  AND  PLANNED  SOLUTION 


ALTERNATE 
SOLUTIONS 


DAY 
TIME 


MAN  : 
HOUPSiERROR 
/STEP!  * 


COMMENT 


Soecify  Reading  and  Writing 

of  Incut  Cards : 
-Number  of  incut  cards  is  de* 
oenaent  on  CARDLIMIT  and  see- 
cified  length  of  character 
String. 

It  is  cal cul atea: 
Number  of  incut  cards: 
(length  of  String  -l)/card- 

limit  +  1 
-Maintain  a  card  counter 
-Read  and  write  each  card  or« 
ceedea  bv  the  acopooriate 
card  number . 


Data  Des  i  gn : 
-Use  a  text  buffer  (Text) 
(String  Array  Indexing  from 
through  256) 


-Use  an  I/O  buf f er (Cardbuf f er ) 
-Use  two  integer  arrays  to  re- 
cord begin  ana    end  oositions 
of  all  palindromes. 

(Begin-of-Dal idrome»End-of- 

p  a  1 indrome) 
-Use  other  self  documenting 

common  data: 

(Caralimit,  Leng t h-o f - t ex t , 

Buf f eroos i t i on ,     Cardcounter, 

Pal i ndromecoun t er ) 
-Use  Index  variables  IX, Jx 
-Use  Local  variables  as  needec 


1/28 
1400 


.5 


1/28 
1500 

1/28 

1500 
1505 


.5 


1/28 

1530 


will  be 
redes  i gneo 
du  r  i  ng 
des  i  gn 
rev  i  ew (D3 ) 


Remarks : 

Error  #1  was  made  during  step  2, 
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ANNEX  A 


Page  3   of  4 


WORKSHEET  FOR  DESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  A 


|  , 

!  MAM 

■ 

STEP 

!  PROBLEM  AND  PLANNED  SOLUTION 

!  ALTERNATE 

[DAY. 

;hours!error :  comment 

tt 

!  SOLUTIONS 

!TI^E 

!/STEP;   *   ! 

5 

Find  Pali  ndromes : 

!  1/28 

-Scan  text  from  left  po  right. 

-Scan 

text 

f  rom 

!  1530 

If  palindromes  of  minimal 

1  eft 

to  r  i 

ght  . 

length  (i.e.  length  is  2  or  3) 

Cons  i 

der  e 

very 

are    found,  call  a  subroutine 

character 

as  the 

which  checks  whether  or  not 

DOSS  i 

b 1 e  oegi n- 

the  oalindrome  is  the  center 

n  i  na 

of  a 

ca 1  in- 

.3 

of  a  larger  oalinarome. 

drome 
Check 

DOSS  i 
goi  ng 

for 
bl  e  1 
back 

al  1 

engt  hs 
ward 

f  rgm 

the  era  of 

the  text  String 

down 

to  a 

string 

1/28 

1 engt  h  of 

2. 

1550 

0 

Recoroing  of  Palingromes: 
-Record  a  oalindrome  only  if 
it  is  not  entirely  included  in 
any  of  the  previously  recordec 
oa 1  i  ndromes . 

-It  is  also  necessary  to  check 
whether  any  of  the  oreviously 
recorded  oalindromes  is  en- 
tirely included  in  the  palin- 
drome last  aetected. 
-A  flag  should  be  set  to  make 
sure  that  only  maximum  length 
oalinaromes  will  be  orinted. 

1/28 

1550 

1/28 
1&20 

.5 

7 

write  all  Palindromes: 
-Identify  the  Dosition  of  each 
palindrome  by  begin  and  end 
with  resDect  to  incut  card 
number  and  position  on  the 
card .                            j 
-Using  the  Dosition  in  the 
text  string  and  the  oarameter  ! 
CARDLIMIT  the  incut  card  num-  ! 
ber  and  the  desired  character  ! 
oositions  can  be  calculated.   ! 

1/28 

lb20 

1/28! 

1630 : 

.2 

Rema  r k  s : 
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ANNEX  A 


Page  4   of  a 


WORKSHEET  FOR  DESIGN  PHASE  AMD  DESIGN  REVIEW  PHASE  OF  PROJECT  *    1 


STEP 

n 


PROBLEM  AND  PLANNED  SOLUTION 


ALTERNATE 
SOLUTIONS 


!  MAN  ! 

day  ;hours:error 
times/step!  » 


COMMENT 


Des  i  gn  Review: 
-Input  cards  will  be  stored 
into  TEXTCstring  array)     con- 
sidering CARDLIMT. 

-Preceeding  the  writing  of  Che 
recorded  oa  1  i  ndromes  a  prooer 
headline  should  oe  printed. 
(TEXT2) 

-If  no  oalinaromes  have  been 
foundi  write  eauivalent  mes- 
sage. (TEXT3) 

-Design  of  printout  format. 


1/28 
2100 


2115 


1/28 

2200 


1.0 


Forgot  ten 
action  in 
step  2 

Forgotten 
ac  t  i  on  in 
steo  6  (D3) 


See  remarks 


Remar *s  : 

Design  of  Printout  Format! 

Palindrome     Begin  End 

Number     Card   Character     Care   Character 
Number   Position    Number   Position 


n         one 
<oa 1  i  nd r ome> 


bco 


flhere  n  is  the  seauence  number 

Den  is  the  cara  number  of  oalindrome  begin 

bep  is  the  character  position  of  oalindrome  begin 

ecn  is  the  card  numoer  of  palindrome  end 

ecp  is  the  charcter  position  of  oalindrome  end. 
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Annex  B 


Program  Listing  of  Project  U     1 
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ANNEX  C 


Pace  1  of  1 


WORKSHEET  FOR  CODING  PHASE  OF  PROJECT  * 
Beginning  of  Coding  (day/time)  :  1/28/2200 
End  of  Coding    (day/time)  :  1/31/1220 
Man  hours  :  7.0  (including  punching  of  cards) 


COOING 


BEGIN 
DAY/TIME 


END 
DAY/TIME 


PROGRAM  PART 


1) 

ER°0R!0AY 
*   {TIME 


COMMENT 

(incl.  coded  error  types) 


1/28/2200 


1/30/1335 


1/30/1615 


1/30/1700 


1/31/0930 


1/28/2300 


1/30/1530 


1/30/1 700 


1/30/1730 


■Oat  a  Definition 
■Initial ization 


■Ut  i  1  i  t  i  es 
(texts,  1/0 
Suorou t  ines) 


■Pali  no  rone 
checx 


Codi  ng  Review 


Punching  Cards 


1/31/1220 


Remarks : 

Man  hours  scent  for  ounching  cards 


5 
6 
7 
8 
9 
10 


11 


12 
13 
iu 
15 


16 
17 
18 
19 
20 
21 
22 
23 
2« 


2205 
2210 


13U0 
laoo 
1U05 
1430 
1500 
1510 


1630 


1710 
1710 
1720 
1725 


1030 
1050 
1110 
1115 
1  125 
1125 
1130 
1135 
1  luo 


1)  Record  when  error  is 
aet  ec  t  ea. 


C23 
C23 


C10 

C8 

C6 

C23 

C23 

D3 


CI  Mi sunaerst anai ng 
o  f  des  i  gn 


Cll 
Cll 
C  17 

C27 


C15 

Cu 

C20 

C28 

C12 

CI" 

C24 

C12 

C24 


(wrong  indentation) 
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ANNEX  0 


Page  1  of  a 


WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  *    :  1  0E8UG  Run  tt    :  1 

Begin  of  Debug  Run  (day/time)  :   1/31/1334 
End  of  Debug  Run    (day/time)  :   2/04/1200 

*  of  Debug  Steos  i nc 1 .  in   Debug  Run:   3    CPU  time  for  Debug  run  (sec) 
CPU  time  for  necessary   comoiles  (sec)  :  5.31 

a)   1 .69    b)   1  .49    C)  2. 13    d)         e)  t )  g) 

Man  hours  for  this  DeOug  Run  :  .9  (exc)uoing  overhead) 


.02 


MAN 


1) 


STEP 
tt 


PROGRAM 

PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY 
TIME 


HOURSJERROR 
/STEP!   3 


COMMENTS 
AND  CODED 
ERROR  TYPES 


Al  I 
oar  t  s 


Get  errorfree  comoile 


3  c  omo  i  1  e 
errors 


Al  1 
oar  t  s 


Al  1 
oar  t  s 


Get  errorfree  comoile 


Get  errorfree  comoile 


2  comoi 1 e 
errors 


O.K. 


1/31 
1335 


1/31 
1U00 

1/31 
180b 
1/31 

1330 

2/04 
1121 
2/04 
1200 


.3 


25 
26 
27 
28 
29 
30 
31 
32 


33 
34 


35 


1  )  Record  when 
error 
occurs 

Al 

C5 

C12 

A2 

C23 

C12 

Cll 

Al 


C23 
A2 


C28 
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ANNEX  0 


Page  2  of    4 


WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  *  :  1  DEBUG  Run  »     :  2 

Beqin  of  Debuq  Run  (day/time)  :    2/05/1330 
End  of  Debug  Run    (day/time)  :    2/05/1500 

#  of    Debug  Steps  i  nc  1  .  in   Debug  Run:   6    CPU  time  for    Debug  run  (sec):  .08 
CPU  time  for  necessary   compiles  (sec)  :  4.99 

a)   2.11    b)   1.46    c)   1.42    d)         e)  f)         a) 

Van    hours  for  this  Debua  Pun  :  1.7  (excluding  overhead) 


STEP! PROGRAM 
a  J   PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


v.  AN 


1  ) 


DAY  JHQURSiERPQR 
TIME! /STEP!   a 


CCMVEMTS 
AN0  CODED 
ERROR  TCPES 


U  t  i  1  i 
t  i  es 


Ut  i  1  i 
t  i  es 


U  t  i  li 
ties 
Ut  i  1  i 
ties 


Ut  i  I  i 

ties 


Ut  il  i 

ties 


Test  all  Texts  exeot 
TEXT3  (all  texts  shou 1 d 
be  crinted  as  designed  ) 


Reoeat  step  1 


Repeat  steo  1 

Test  spacing  and  printing 
of  b 1  an  k  lines 
(all  formats  snould  ap- 
pear as  oes  i gned) 
Test  calculation  of  num- 
ber of  inout  cards  for 
minmum  length  of  inout 
string  (2  characters) 
(oroqram  should  read  on- 
ly one  card) 

Test  detection  and  writ- 
ing of  a  palindrome  of 
mini  mum  length 
(oalinorome  should  be  de- 
tected and  listed  as  de- 
s  i  gned) 


Run 
error 


Run 
error 


Spac  i  ng 
orob 1  ems 


0.K  . 


O.K. 


O.K. 


2/05 
1345 

2/05 
1420 

2/05 
1420 

2/05 
14U5 

2/05 

1445 


2/05 
1500 


36 


37 
38 
39 


40 


1  )  Record  when 
error 
occurs 
01  1 


C8 

C27 

C20 


C28 
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ANNEX  D 


Page  3  of  t 


WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  u    :  1  DEBUG  Run  U    :  3 

Begin  of  Debug  Run  (day/time)  :    2/05/1500 
End  of  Debug  Run    (day/time)  :    2/05/1600 

*  of  Debug  Steps  i nc 1  .  in   Debug  Run:   5    CPU  time  for  Debug  run  (sec):  8.79 
CPU  time  for  necessary   comoiles  (sec)  :  3.07 

a)   1.51    b)   1.56    c)         a)        e)  f)        g) 

Van    hours  for  this  Debug  Run  :  .R  (excluding  overhead) 


VAN 


1) 


STEPIPR0GRAM 
u     J   PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY  '.HOURS  !  ERROR 
TIMEi/STEPi   a 


COMMENTS 
AND  CODED 
ERROR  TYPES 


Ut  i  I  i 
ties 


Pal  in- 

drome 

Check 


Pal  in- 

arome 

check 


Pal  in- 

drome 
check 


Test  calculations  of  num- 
ber of  incut  cards  for 
maximum  string  length 
(i.e.  256  characters) 
(program  should  rear)  4 
cards ) 
Test  detection  ana  writ- 
ing of  a  oalidrome  of 
ma x  i  mum  1 engt  h 
(PALIDROME  COUNTER  should 
be  1,  palidrome  shoula  be 
detected  and  orinted  as 
desi  gned) 


Reoeat  steps  1  and  2 
Test  detection  and  writ- 
ing of  maximum  number  of 
palindromes  of  minimum 
1 engt h 

(PALINDROME  COUNTER 
should  be  128,  all  128 
palindromes  should  be 
listed  as  designed) 
Test  for  string  without 
oa 1  i  ndr omes 
(TEXT3  should  be  printed) 


O.K 


run 
error 


O.K. 


O.K. 


TEXT3 
not  prin- 
ted 


2/05 
1500 


1500 


2/05 
1530 

2/05 

1530 


1530 
1530 


.5 


.  u 


Remarks ! 


ai 


U2 
U3 


'run  error"  indicates  termination  due  to 


2/05 
1600 
excess  of  array  limits 


1  )  Record  when 
error 
occurs 


C28 


83 
C2t 
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ANNEX  0 


Page  u  of  a 


WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  *  :  1  DEBUG  Run  U    :  a 

Begin  of  Debug  Run  (day/time)  :    2/05/1600 
End  of  Debua  Run    (aay/time)  :    2/05/1700 

tf  of  Debug  Steps  incl.  in   Debug  Run:   2    CPU  time  for  Debug  run  (sec):  .22 
CPU  time  for  necessary   comoi les  (sec)  :  1.67 

a)   1.67    b)  c)  c)         e)  f)         g) 

Van    hours  for  this  Deoug  Run  :  .5  (excluding  overhead) 


WAN 

1 

I 

STEP 

PROGRAM 

OBJECTIVE  AMD    EXPECTED 

ACTUAL 

DAY 

HOURS IERR0R 

COMMENTS 

u 

PART 

RESULT 

RESULT 

TIME 

/STEP!   s 

1 

AND  CODED 
ERROR  TYPES 

1)  Record  wnen 

error 

occurs 

1 

Initi- 

Test for  inout  of  illegal 

2/05 

.5 

al  i  za- 

S  t  r  i  ng  1 eng  t  h 

O.K. 

1600 

t  i  on 

(program  should  print  a 
-arning  and  terminate) 

2 

Pal  i  n- 
drome 
chec  k 

Test  for  overlaooing  oa- 
1  i  ndromes 

(should  be  recorded  as 
two  oa 1 i  ndromes ) 

O.K. 

2/05 

1630 
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ANNEX  El 


Page  1  of  2 


ERROR  LISTING 
PROJECT  a    :  1 

Begin  of  Project  (day/time)  :  1/28/1040 
End  of  Project  (day/time)  :  2/06/1620 
Man  hours  for  total  project  :  21.8 


ERROR 


PHASE 

in  which 

ERROR  was 

di  s- 

coveced 


PHASE 
in  which 
ERROR  was 
*ade 


ERROR 

TYPE 

(see  ANNEX  F) 


TIME 
scent  to 
solve  the 
ERROR 
(."an 
■n  i  n  .  ) 


*     of  OTHER 
STATEMENTS  OP 
PARTS  OF  THE 
PROGRAM 

AFFECTED 


a 

5 
6 
7 
8 
9 
10 

1  1 

12 
13 
la 
15 
16 
17 
18 
1R 
20 
21 
22 
23 
2U 

25 

26 
27 


Oes  i  gn 
Oes  i  gn 
Coding 


Cod 
Cod 
Cod 
Coa 
Coa 
Cod 
Cod 

Cod 

Cod 
Cod 
Coa 
Cod 
Cod 
Cod 
Cod 
Cod 
Coa 
Cod 
Coa 
Coa 
Coa 


ng 

ng 
ng 
ng 
ng 
ng 
ng 

ng 

ng 
ng 
ng 
ng 
ng 
ng 
ng 
ng 
ng 
ng 
ng 
ng 
ng 


Oebuggi  ng 

Debugqi  ng 
Oebuggi  ng 


Oesi  gn 
Oes  i  gn 
Coding 


Coa 
Coa 
Coa 
Coa 

Cod 
Coa 

Des 

Cod 

Coa 
Cod 
Cod 
Cod 
Coa 
Cod 
Cod 
Coa 
Coa 
Coa 
Cod 
Cod 
Cod 

Cod 

Cod 
Cod 


ng 
ng 
nq 
ng 
ng 
ng 
gn 

ng 

ng 
ng 
ng 
ng 
ng 
ng 

ng 
ng 
ng 
ng 
ng 
ng 
ng 

ng 

ng 
ng 


03 
03 
C23 


C23 

C10 

C8 

C6 

C23 

C23 

03 

CI 

Cll 

CI  1 

C17 

C27 

C15 

Cu 

C20 

C23 

C12 

C19 

C24 

C12 

C2« 

Al 

C5 
C12 


i 
2 

5 

5 

5 

10 

30 

10 

5 
3 
5 
5 
5 
5 
5 
5 
5 
5 

10 
3 

15 

10 

5 
5 


Data  incut  only 
Data  orintout 
Index i  ng  within 
subseauent  sub- 
routine 


3  s t  at ement  s 
Whole  subrou- 
tine af  fee t  ec 
Whole  subrou- 
tine affected 


rihole  subrou- 
tine a  f  fee  tea 
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ANNEX  El 


Page  2   of  2 


ERROR  LISTING 
PROJECT  *  :  1 

Begin  of  Project  (day/time)  :  1/28/1040 
End  of  Project  (day/time)  :  2/06/1620 
Man  hours  for  total  project  :  21.8 


ERROR 

u 


PHASE 
in  which 
ERROR  was 
dis- 
covered 


PHASE 
in  which 
ERROR  was 
made 


ERPOP 

TYPE 

(see  ANNEX  F) 


TIME 
soent  to 
solve  t  h< 
EPROD 
(Van 
m  i  n  .  ) 


*    of  OTHER 
STATEMENTS  OR 
PARTS  OF  THE 
PROGRAM 

AFFECTED 


28 
29 
30 

31 
32 
33 
34 
35 
36 
37 

38 
39 

40 

Ul 
42 
43 
44 


Debuggi  nq 
Debugai  ng 
DeDuggi ng 

Debuggi  ng 
Debuggi  ng 
Debuggi  ng 
Debuagi  ng 
Debuggi  ng 
Deouggi  ng 
Debugg  i  nq 

Debuagi  ng 
Debuggi  ng 

Debuggi  ng 

Debuggi  ng 

Debugg  i  ng 

Debuggi  nq 

Test  i  ng 


Con  i  ng 
Codi  ng 
Codi  ng 

Cod  i  ng 
Codi  ng 
Cod  i  ng 
Cod  i  ng 
Des l gn 
Des  i  gn 
Cod  i  ng 

Codi  ng 
Cod  i  ng 

Codi  ng 

Codi  ng 

Debuggi  ng 

Cod  1 ng 

Des  i  gn 


A2 

C23 

C12 

CM 

Al 

C23 

A2 

C28 

01  1 

C8 

C27 
C20 

C28 

C28 
B3 
C28 
09 


5 
3 
5 

3 

3 

5 

5 

10 

25 

10 

5 

10 

20 

15 
3 

15 
20 


Writina  of  pa- 
1  i  ndromes 

Writing  of  pa* 
1 i  ndr owes 
Whole  subrou- 
tine a  f  fee  ted 
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ANNEX  £2 


Page  1  of  2 


ERROR  LISTING  CCOMMENTS) 


ERROR 

M 


DAY 
TIME 


COMMENTS 

(EVIDENCE,  THOUGHTS,  WHY  «AS  THE  ERROR  MADE? 

WHY  AND  HOW  WAS  THE  ERROR  DISCOVERED? 

ERROR  BLOCKING,  etc.) 


5 
6 

7 

a 
q 

10 

1 1 

12 
13 

la 
15 

16 
17 


1/28 

1505 
2115 

2205 

2210 
1/30 
13U0 

laOO 
ia05 
l«30 

1500 
1510 
1630 
1710 
1710 
1720 
1725 
1/31 
1030 
1050 


18 

1110 

1<? 

1115 

20 

1120 

21 

1  125 

22 

1  130 

23 

1135 

2a 

1  tao 

1/31 

25 

1335 

26 

1335 

27 

1335 

28 

1335 

29 

1335 

30 

1335 

31 

1335 

32 

1335 

Lack  of  concentration  (steo  left  out,  which  the  programmer 
was  aware  of  ) 

This  design  steo  was  costooned  and  forgotten  later  on. 
Programmer  dia  not  c^ecu  the  manual.   (Discovered  while 
examining  the  solution  of  a  similar  oroDlem  in  an    old  program, 
s  ame  as  3 


Error  was  found  while  examining  the  list  of  error  tyoes 
in  connection  with  errors  3  ana  a. 

•  hile  reacting  the  orogramming  manual. 
discovered  during  a  cesk  test. 


Error  was  founa 

Errors  10  anr)  11 

see  1 0 

(wrong  indentation  could 

see  12 


ead  to  orogramming  errors) 


This  statement  was  inserted  between  the  wrong  lines. 

It  was  aetectea  in  tnis  early  stage  of  the  oroject 

because  the  programmer  was  ounchina  earns    himself,  which 

allowed  him  to  review  his  source  statements. 

Programmer  did  not  look  at  the  declaration  of  the  variaole 

(missing  mandatory  declaration) 
(wrong  format) 

same  as  20 


special  character  ( " )  was  expected  to  be  necessary  due  to 

similarity  with  other  uses  of  .iRITE/WRITEON,  however  this 

assumption  was  wrong 

(previously  declared  variable  declared  twice) 

same  as  20 

Programmer  did  not  check  the  programming  manual, 
see  20 
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ANNEX  £2 


Page  2   of  2 


ERROR  LISTING  (COMMENTS) 


ERROR 

!  DAY 

|  COMMENTS 

» 

!  TIME 

,  (EVIDENCE,  THOUGHTS,  WHY  WAS  THE  ERROR  MADE? 
!   why  AND  HOW  WAS  THE  ERROR  DISCOVERED? 
I   ERROR  BLOCKING,  etc.) 

1/31 

33 

1806 

3a 

1806 
2/04 

Error  could  Save  been  avoided  by  looking  at  previous  block  o 

f  code. 

35 

1122 
2/05 

Porgrammer  had  correct  design  conceot  but  did  not  coae 
correct ! y . 

36 

1345 

Boundary  conditions  had  not  been  chec*ea  before. 

37 

1420 

38 

1420 

39 

1420 
2/05 

40 

14a5 
2/05 

Trivial  error  was  ti  a  d  e  »  cecause  orogrammer  was  tired. 

41 

1500 

Design  was  not  stated  well. 

42 

1530 

same  as  40 

43 

1530 

2/06 

■ 

ua 

1525 
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ANNEX  F 


FINAL  STATISTICS 


PROJECT  *   1 

FINAL  STATISTICS 

Project  name  :  PALINDROMES 

Short  desc  r  i  ot  i  on : 

Given  an  inout  String  of  length  n,  -here  2  <-  n  <-  25e» 
fina  all  occurences  of  oalindromes  and  list  tiem  with 
reSDect  to  their  beginning  and  ending  positions  on  the  inout 
card. 

Inout:  via  ounch  cards 
Outout:  via  line  orinter 

Quantitative  measures: 

1.  8    of  source  statements  :  141 

2.  Total  man  hours  for  oroject  :  21.8 

3.  Man    hours  spent  in 
a)    Oes  i  on  :   5.0 

D )  Codi  ng  :   7.0 

c )  Oebuggi  ng  :  4.0 

d)  Test  i  ng  :  5.8 

4.  CPU  time  for  comoiles:  20.22    sec. 

5.  CPU  time  for  aebug  runs:  P. 11  sec. 
o.  CPU  time  for  test  runs:  13.98  sec. 

7.  u    of  test  and  deoug  runs:  5 

8.  u    of  test  and  debug  steos:  23 

9.  tt    of  errors  found:  4a 

10.  Total  man  hours  used  to  correct  errors:  6.73 
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ANNEX  F 


FINAL  STATISTICS 


11.  Error  Detect  i  on: 
a)  Mean  time  between  error  detections:  20.3  man  min. 


*    of 
Errors 
T 


2U 


20  + 


10  ♦ 


5  ♦ 


****** 


******* 
*      * 


******* 
*      * 


******* 

*      ******* 


*      * 

************* 
____-+_-___+--_-_+__.__+...__♦_-__■_+_«___+_____+ 

0  -15  16-30  3 1  -a5  46-60  61-75  76-90  91-105  106-130 


****** 

151-175    man  min. 
of 
project  time 
TIME  BETWEEN  ERROR  DETECTIONS 
(measured  from  coint  in  time  of  detection  of  previous  error) 
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ANNEX  F 


FINAL  STATISTICS 


12.  Error  Correction: 

a)    Mean  time  to  correct  an  error:  7.8  man  m i n , 

8  of 
Errors 
correct  ed 

T 


20* 


15* 


10  + 


5  + 


2  3  a  5 


10 


15 


20 


25         30 

man  minutes 
of  croject  time 


TIME  TO  CCRPECT  ERRORS 
(measured  from  point  in  time  of  detection) 
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ANNEX  F 


FINAL  STATISTICS 


13.  When  errors  were  f  o  u  n  a  : 


a)  *  of  errors  found  during  design  phase: 

b)  *  of  errors  found  during  design  review 

c)  *  of  errors  found  during  coding: 

d)  *  of  errors  found  during  deDugaing: 

e)  •  of  errors  found  during  writing  of 

test  orocedures : 

f)  9  of  errors  found  during  testing: 


1 
1 

22 

19 

=  2.3 
=      2.3 

=  50.0 
=   a3.2 

X 
X 
X 
X 

0 
1 

=  0.0 
=      2.3 

x 

X 

1U.  ft  hen    errors  were  mane  l 


aa 


a)  *  of  errors  made  during  design  phase: 

o)  >t  of  errors  itade  during  design  review: 

c)  *  of  errors  maae  auring  coding: 

d )  "  of  errors  made  durinq  debugaing: 

e)  t  of  errors  made  during  «ri  t  ina  of 

test  orocedures : 

f)  s  of  errors  marie  durira  test  ina: 


5 

0 

38 

1 

0 
0 


1  1  , 
0, 

86, 
2. 


0.0  % 
0.0  X 


aa 
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ANNEX  F 


FINAL  STATISTICS 


15.  TIME   HISTORY  GRAPHS 


»    of 
Errors 

t 


8  ♦ 


5  ♦ 


1  * 


C 

c  c 


A 

c 
c 

A 

c 
cc 


CCC  CA   C  B 


Error  T yoes : 
A:  Clerical  error 
B:  Deougginq  error 
C :  Codi  nq  error 
0:  Desian  error 


D 


0C  CCCCCCC   CCC  ACDCCCC 


1  2  3  a  5  10         15         20 

.Design. , .Coding. Debuqgi  nq. .Test  i  ng, 

(Software  Develooment  Phases) 


man  hou r s 


NUMBER  OF  ERRORS  FOUND  VS  PROJECT  TIME 


U    of 
Errors 


5  + 


*  *    *  * 

*****  •*** 


* 
* 
*  *  * 
*********** 

************ 


1  2  3  a  5 


10 


15 


•  +  -• 
20 


■nan  nours 


NUMBER  OF  ERRORS  CORRECTED  VS  PROJECT  TIME 


3  - 


ANNEX  F 


ERROR  CATEGORIES  AND  TYPES 


Oes  i  gn  Errors 

The  follwing  types  of  errors  aeoly   to   both   categories 
"System  Design  Errors"  and  "Program  Oesign  Errors": 


01 

02 
03 

on 

D5 

D6 

07 

08 

0"? 

010 

011 

012 

013 

D14 

015 

016 

D17 


Communication  Error 

Design  Negligence 

Forgotten  Cases  or  Steos 

Timing  Problems 

Errors  in  1/0  Concepts 

Oata  Design  Error 

Initialization  Error 

Inaaeauate  Checking 

Extreme  Conditions  Neglected 

Seauencing  Error 

Indexing  Error 

Looo  Control  Errors 

Misuse  of  Boolean  Expression 

Mathematical  Error 

ReD r esen t a r i on  Error 

Misunaerstanding  of  P  r  o  D  1  e  m  Specifications 

Other  Oesian  Errors 


Coding  Errors 


CI 

M  i  sunder 

C2 

Neg 1 i  gen 

C3 

:   1/0  Form 

C4 

Mi sp 1  ace 

C5 

:   Mu 1 t  i  ol e 

C6 

:   M i  ss i ng 

C7 

:   Inadeo.ua 

C8 

Initial i 

C9 

Error  in 

C10 

Inadeaua 

CI  1 

Level  Pr 

C12 

M  i  s  s  i  n  a 

C13 

Case  se 1 

Cia 

GO  TO  Pr 

CIS 

Comnen  t 

C16 

Forgot  t  e 

C17 

Incons  i  s 

C18 

wrong  Us 

CIO 

I ndex  i  ng 

C20 

Incons  i  s 

C21 

Seauenc  i 

C22 

F 1 ag  Usa 

C23 

Synt  ax  E 

C24 

Looo  Con 

C25 

I ncor rec 

C2o 

Language 

standirg  of  Design 

ce 

at  Error 

d  Data  Dec  1 arat  i  on 

Data  Dec  1 arat i ons 
Data  Declaration 
t e  Data 
zation  Error 

Parameter  Passing 
te  or  Forgotten  Checking 
oD 1  ems 

Declarations  of  310c1*  Limits 
ec  t  i  on  error 
ob 1  ems 
Error 

n  Delimiter 
t  enc  y  in  Mam  i ng 
e  of  Nested  IF  Statements 

Error 
tent  Use  of 
ng  Error 
ge  Prob 1  ems 
r  ror 

t  ro 1  Error 
t  Exit  from 


Variables  or  Data 


SuOrcut  i  nes 


Usage  Pron 1  ems 
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ANNEX  F 


ERROR  CATEGORIES  AND  TYPES 


C27:  Forgotten  Statements 

C28:  Representation  Error 

C29;  Control  Seauence  Error 

C30:  Incorrect  Subroutine  Usaae 

C31:  Other  Coding  Errors 


3.   Clerical  Errors 


Al 
A2 
A3 

AU 


Manua 1  Error 
Ment al  Error 
Procedural  Errors 
Other  Clerical  Errors 


4.   Debugging  Errors 


31  :  I  naoo roor i a t e  Use  of  Deougging  Tools 

32  :  Insufficient  or  Inaooroor i at e  Selection 

of  Test  Cases  or  Test  Data 

33  :  Misinterpretation  of  Debugging  Results 
84  :  M i s i n t eror e t a t i on  of  Error  Source 

35  :  Meg  1 i  aence 

8o  :  Other  Debugging  Errors 

Testing  Errors 


Tl 
T2 
T3 
TU 
T5 


Inadeauate  Test  Casefs)  or  Test  Data 

Misinterpretation  of  Test  Results 

Mi s i n t ere  re t a t i on  of  Program  Specification 

Neg 1 i  gence 

Other  Testing  Errors 
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ANNEX  G 


Page  1  of  9 


DIRECTED  GRAPH  REPRESENTATION 
PPOJECT  a  :   i 
Program    Dart  :  INITIALIZATION 

COMPLEXITY  MEASURES: 
NUMBER  OF  STATEMENTS:  la 


NUMBER  CF  NCCES:  5 

NUMBER  CF  ARCS  :  5 

NUf-BER  Ce  PATHS:  2 

CVCLCMATIC  NUMBER:  7(G)* 


REACHABILITY  CF  NCDES 


NCCE  L 

NGCE  2 

NCCE  2 

NuCE  4 

NCCE  5 


SUM:       7.CCCC0C 
REACHABILITY  INDEX 
QF  OIR5CTEC  GRAPH: 


1.AO0OOJ 


«-  Error  s     37 
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ANNEX  G 


Pace  ?  of  ? 


DIRECTED  GRAPH  REPRESENTATION 
PROJECT  =  :   1 
Progra"  Dart  :  BLANK  LINES 


COMPLEXITY  MEASURES: 
NUMBER  OF  STATEMENTS:  o 


a 

6 

V  (  G  ) 


i\upr  =  R 

Cf 

NCCES 

NUr"?ER 

CF 

ARCS 

NUfSER 

CF 

PATHS 

CYCLCMATIC  NUMBER 


SE^ChAEILITY  CF  NCCSS 


NCCE 

1 

:            I 

NGCE 

2 

:            I 

NCCE 

i 

1 

NCCc 

H 

:           2 

NCCE 

c 

1 

NCCE 

I 

2 

SUP:  S.CCCOCC 

SEJCi-AB  IL  ITY     INDEX 
Cc    DIR5CTEC    6R£PH: 


122333 
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4  N  N  E  X  G 


Paae  5  of  " 


DIRECTED  graph  REPRESENTATION 
PROJECT  a  :   1 
Program  oar?  :  T&XTt,  TEXT2,  TExT^ 

COMPLEXITY  MEASURES: 


•-IUMBEk  OF  STATEMENTS 

NUVSEP  CF  \CCSS: 
NUNSER  Cf  J^CS  : 
MJ>6cP  CF  fiThS: 
CYCLCrMTIC  iN  U  >  3  E  ?  : 


I 
V(C-) 


1 1 


^EiCHAeiLITY  CF  NCDE5: 
NQCc     I   :     1 
NuCE     2   :     1 


SUN:       2.CCCCCC 
?£AChiP TL  ITY  INCEX 
CF  DIRECTEC  GRAPH: 


;C0GG 
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ANNtX  G 


p  33<?  U  of   ^ 


OIPECTEO  GRAPH  FEpPE3EhTi T 10' 
PROJECT  «  :   l 
Proaram  oart-  :  3£AQ  AND  .iRITE  INPUT  CA*OS 

COMPLEX  IT r  MEASURES: 


NUVbEH  CF  STATEMENTS 

MJPSSR  CF  NCC5S 
NUMBER  CF  ARCi 
NURSES  CF  PATHS 
CYCLCKATIC  NU.*3 


:     18 

o 

7 

v(G) 


SEiCI-ASILITY    CF    NOCES 
1 


NGCS 

1 

NCCE 

2 

NCCS 

1 

NCCS 

A 

NCCE 

5 

NCCE 

6 

SUP:  Z'.CCCCC 

SE^CrASIl  ITY     INCEX 
CF    0  IRECTEC    QtHP\- : 


itJJJJjJ 
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ANiMdX     li 


r-aoe    5    of    <) 


DIRECTED  G«iph  REPRESENTATION 

PROJECT  a  :   i 

Program  oart  :  .i^ITE  ALL  P  A|_  I  •^DROr-'ES 


Error  -  36 
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ANNEX  G 


Paqe  b  o  f  Q 


DIcECTE0  GH^PH  ^EBC!io£NTJTION 
PROJECT  =  :   I 
Program  oart  :  PALINDROME  CHECK 


a )  -    of  nodes:  10 

b  )  2  0f    arcs     :  15 

c)  s    or  sratenenrs:   7 

a)     -    of  oaths :  * 

e)  'escaoi  Mtv:  ♦ 

f)  Cvclo^aric  numosr:  5 
*  Number  of  cat'is 

ana  reacnaDi  I  i  fv 
are  very  la  roe. 
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AMMfcX  G 


Page  7  of  <? 


DIRECTED  GRAPH  ^EPPE SENT  A T I  ON 
PPGJECT  s  :   l 
Program  oart  :  CONTINUE  CHECK  IMG 

COMPLEXITY  MEASURES: 


NUMBER  CF  STATEMENTS:  IS 

NUfBE?.  CF  NCCES:  3 

MIPS 5R  CF  ABC S  :  9 

MlfrSER  CF  PATrS:  7 

CYCLCNATIC  NUfSEF  :  V (G)  = 


RSAO-A8ILJTY  CF  NCSSS 


NOCE 
NCCE 
NOCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCc 


SUM:  iC.CCCQO 

SEACt-A6lL  ITY     INCcX 
CF    CIr-.ECT5C    GRAPH: 


z «  CCC^C^ 
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A  N  i\i  £  X     G 


p a c e    8    of    9 


DIRECTED    liRAP"    REPRESEMTATION 
PROJECT    a     :       1 
Proaram    oart     :     RECORD    P4Llr>iDR0.yE 


a)  e    of    no^es :    Q 

b )  a    of    arcs     :     12 

c)  =    of    stste'ents:    21 

d)  -    of    oatns:     * 

e)  ceacnaoility:     * 

f)  CyclomatlC     nuTicer:     5 
*    'lufloer    of    satis 

and    reac  h  ao i  1  1  t  y 
are    very     lane. 
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ANNEX  G 


Paoe  9  of  <? 


DIRECTED  GRAPH  REPRESENT* r  ION 
PROJECT  s  :   1 
Program  oart  :  MAIN 

COMPLEXITY  MEASURES: 


NiJVHER  OF  STATEMENTS:  c 

,NUN"5ES  CF  NCCES:  6 

NOYecS  CF  ASCS  :  6 

iNUfBEfi  CF  PATHS:  2 

CYCLCYATIC  NUYEE?:  V(G>  = 


R6ACHA8  ILITY 


NGC5 
NQCE 
NCCE 
NCCE 
NGCE 
NQCE 


CF  NCCES : 
1 
1 
1 
1 
2 
2 


SLY:  5.CCQ0CO 

REACHABILITY     INCEX 
CFCISECTECGPAPp-:  l 
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ANNEX  H 


Page  1   of  2 


TEST  PHASE  DESCRIPTION 


Project  *  :  1 

Test  run  ft     :  1  Including   7  Test  Steos 

Begin  of  Test  (dav  time)  :   2/06/1430   End  of  Test  (day/time)  :   2/06/1620 


CPU  time  for  necessary   comoiles  (in  sec.):  5.18 
a)   1  .6^   b)   1.71c)   1 .78  d)       e)         f ) 


g) 


CPU  time  for  TEST  run  (sec)  :  13.98 

Man  Hours  in  Testina  :  5.8  (including  preparation  of  tests) 


TEST 
STEP 


OBJECTVE 


EXPECTEO 

RESULT 

(TOLERANCE) 


ACTUAL 
RESULT 


1  ) 

ERROR 


DAY 
TIME 


COMMENTS 
AND 

CODED  ERROR 
TYPES 


Check 
sma  I  1 

(a)  d 
1 

(b)  p 
1 

Check 
pal  i  d 

(a)  d 
1 

(b)  p 
1 

Test 
Repea 


progr 
oalin 
a  1  i  ndr 
engt  h 
a  1  i  ndr 
engt  h 

for  1 
romes 
a  1  i  ndr 
engt  h 
a  1  i  ndr 
engt  h 
cor rec 
t  s  t«p 


am  for 

dromes 

ome  of 

2 

ome  of 

3 

arge 

ome  of 
255 

ome  of 
25o 
t  i  on 
s  1  ,2 


Test  cor rec  t  i  on 
Repeat  steps  1  , 2 
Check  oa 1  i  ndromes 
which  cross  cara 
boundar  i  es 
Check  for  oalin- 
dromes  of  various 
1 engt  hs  and  sizes 
Check  for  i  nva 1 i  d 
i  nout  (string 
1 engt  h  =  1 ) 


All  oa 1 i  ndromes 
are  detected  and 
recorded  as  soe- 
c  i  f  i  ed  b v  pro- 
gram soec  i  f  i  - 
cations 

same  as  1 


same  as  1 


same  as  1 
snou 1 d  be  de- 
dected  ana    prin- 
ted as  spec  i  f  i  ed 
same  as  5 


Earning  and  ter- 
mination 


O.K 


progr 
error 


j  OD 
t  erm  i ■ 

na  t  ed 

O.K. 


O.K 

O.K, 
O.K, 


U4 


2/06 
1U30 


,525 


1550 


2/06 
1620 


1)  Record  unen 
error  occurs 


D<3 


Imo 1 emen  t  a  t  i  on 
error  (time 
estimate  exceeded) 


See  notes 


Notes; 


Other  invalid  inouts  were  exhaustively  tested  during 
debugging  phase. 
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ANNEX  H 


Page  2      of    2 


Choice  of  Test  Data: 

Some  test  data  were  chosen  to  check  the  program  for  boundary 
conditions.   Other  test  data  were  selected  to  check  random 
size  and  numbers  of  palindromes. 
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APPENDIX  8 
PROJECT  DESCRIPTION 
Project  U    :    2 

Project  title:   PATH  ANALYSIS  IN  DIRECTED  GRAPHS 
Programmer  :  HOFFMANN 
Programming  Language  :  ALGOL 

Programmino  environment:  IBM/ 36 0/67 , OS/MVT / BATCH 

and  TIME  SHARING  (CP/CMS) 

Design  notes  :  see  ANNEX  A 

Program  listing  :  see  ANNEX  B  of  APPENDIX  C 

Coding  notes  :   see  ANNEX  C 

Debugging  notes  :   see  ANNEX  D 

Error  Listino  :    see  ANNEX  E 

Final  statistics  :   see  ANNEX  F 

Graphical  reoresentaticn  :  see  ANNEX  G 

Test  phase  descriotion:  see  ANNEX  H 


Start  i  ng  date :  i  mar  77 


Enainq  aate:  17  M  A  R  77 


EXPERIMENT  DESCRIPTION 

1.  Project  description: 

A.   General  Descriotion 

This  program  is  desiqned  to  find  and  write  all  Dossible 
oaths  of  minimal  lenath  (i.e.  if  a  directed  graph  contains 
loops/  no  loop  may  be  traversed  more  than  once  in  succes- 
sion) through  a  directed  graph. 
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B.   Inout  Oescriot ion: 

(a)  Batch  Processing  : 

Card  1:  1  in  card  column  1  followed  by  any  text 

(this  text  will  identify  the  graph  and  will 
be  printed  as  a  headline  preceding  the  input 
document  at  i  on ) 
Card  2:     t  in  card  column  1   followed  by   one  blank, 
where  t  specifies  the  type  of  oath  listing 
des  i  red . 

t=0  ->  all  paths  are     listed  during  analysis 
(recommended  only  for  a  very  large 
number  of  paths,  i.e.  more  than  50  0 
paths  e  xDec  t  ed ) 
t  is  nonzero  ->  paths  are  listed  after 
analysis  has  been  completed 
(any  additional  information  will 
not  be  interpreted  by  the  program) 
Card  3:  s  in  care  column  1  followed  by   one  blank 
where  s  specifies  the  number  of  digits 
of  the  largest  node  name  used  in  the  directed 
graph  and   1<  =  s  <=3 

(any  aoditional  information  will  not  be 
interpreted  by  the  orogram) 
Card  4:1    n    si    s2    ...    sn 

where  1  is  the  node  name  of  the  entry  node 
and  may  be  punched  in  any  column 
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n  is  the  number  of  successors 
and    0  <=  n  <=  14 
si  ( i  -  1 1 2 ,     ...  n)  specifies  the  name 
of  the  ith  sucessor  of  node  1 
Card  5  thru  Card  m+3: 

where  m  specifies  the  number  of  nodes  in 
in  the  directed  graph 
Format:  j    n    si    s2    ...    sn 

where  j  specifies  the  node  name  of  the  node 
being  described  on  this  card 
and  2<-     j  <=  999;   j  may  be  punched 
in  any  co 1 umn . 
n  is  number  of  successors 
and   0<=  n  <=  14 
si  (i=l>2  ...  n)  specifies  the  name  of 
the  iths  successor  (1  <=  si  <=  999) 
if  n=0  ->  no  entry  necessary 
NOTE: 

(a)  All   entries   are   integers  and  must  be  separatea  by  at 
least  one  blank. 

(b)  All  information  about  any  node  must  P  e  on  one  card. 

(c)  Cara  column  80  must  contain   a   blank   character   or   an 
additional  blank  card  has  to  be  inserted. 

Card  4  +  m:    99999   followed  by  at  least  one  blank 
(this  incut  indicates  that  all  information  of 
the  oreceeding  directed  graph  has  been  input) 
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NOTE:  Following  this  description  any  number  of  directed 
graphs  may  be  specified  and  submitted  to  the  program.  After 
the  last  descriDtion  cf  a  directed  graoh  a  termination  card 
has  to  be  added  which  contains  any  integer  different  from 
"1"  in  card  column  1 . 


Samp  1 e  Input  : 


1 

0 

2 

1 
2 
3 
a 

5 
99999 

1 

1 

1 

1 
2 
3 
a 
5 

99999 
-1 


(FIRST  GRAPH) 
TOGGLE 

MAX  NODE  NAVE 
1     2 

3   a 


(SECOND  GRAPH) 
TOGGLE 

VAX  NODE  NAVE 
3    2 

a 

1    5 


TERMINATE 


(b)  Inout  under  CP/CMS: 

The  input  seauence  under  CP/CMS  is  similar  as  under 
batch  processing.  There  is  only  one  exception:  Any  other 
inout  than  the  inout  requested  by  the  program  will  terminate 
the  orogram.  The  user  can  only  input  integers?  additional 
blanks  are  not  alloweo. 

Under  CP/C^S  instructions  are  displayed  at  the  terminal 
which  makes  the  use  of  the  program  almost  se 1  f -exo 1  ana t or y . 
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NOTE: 

(a)  Login  with  540 k 

(b)  Follow  the  directions  of  the  program  Drecisely. 

(c)  Incut  only  integers. 

C  .  Lrror  Messages : 

All  error  messages  are    sel f-exDl anatorv. 
Errors  3,  8,     9,  10  refer  to  invalid  or  incomplete  input. 
Errors  8  and  9  will  cause  the  program  to  terminate. 
Error  2.       indicates  the  limit  of  the  program.  If  error  2    oc- 
curs it  is  most  likely  that  the  directed  graoh  has  an  infin- 
ite  number  of  paths.   The  user  may  try  the  same  input  using 
"0"  as  input  on  the  second  card.   Error  1  will  only  occur  if 
more   than  30  nodes  nave  been    specified  having  14  successors 
each  . 

All  other  errors  will  indicate  an  error  which  has  not 
been  found  during  testing  and  debugging  or  will  be  due  to 
some  abnormal  usage  of  the  program.  In  either  case  it  might 
be  possible  to  locate  the  problem  by  examining  the  program 
listing  or  retrying  the  program. 
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2.  Programmer's  background: 

a)  Experience  in  programming: 

Oct  1970  -  May   1971    Programming  courses 

May  1971  -  April  1972   Module  Programmer 

May  1972  -  June   1974   rtork  in  Test  and  Simulation  Depart- 

men  t  at  t  He 

NAVAL  COMMAND  AND  CONTROL  SYSTEMS 
COMMAND  (FEDERAL  GERMAN  N/TVY) 
Testing  of  tactical  real  time  systems 

March  1975  -  Jan  1977   Student  at  the  NAVAL  POSTGRADUATE 

SCHOOL,  Monterey,  Computer  Science 

b)  Experience  in  testing: 

Two  years  of  work  in  testing  ana  simulation. 

c)  ExDerience  in  the  area  of  the  given  problem:   None. 

d)  Experience  in  the  proaamming  language  being  used: 
ExDerience   over   a   period  of  18  months  in  more  than  10 

programming  orojects.   (Total  number   of   source   statements 
oroduced  during  that  time  was  more  4000.) 

3.  Psychological  factors: 

a)  Dio  the  proqrammer  like  the  project?   Yes. 
bj  How  does  the  orogrammer  like  the  programmina 
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language. 
Favorite  programming  language. 

c)  Was  the  programmer  satisfied  by  the  way  the  problem 
was  soec  i  f i  ed? 

Only  minor  criticism. 

d)  How  did  the  programmer  like  the  programming  environ- 
ment ? 

The  facilities  (studv  room/  card  punch  room)  were  not 
conducive  to  efficient  orogramming  because  of  restricted 
soacer  bad  lighting  and  noise. 

e)  Other  factors: 

The  recording  of  the  experiment's  data  during  the  pro- 
ject affected  soeed  and  concentration  considerably. 
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4.  Comments  on  Documentation 

For  the  documentation  of  each  software  development  phase  a 
special  documentation  form  has  been  developed.  These  forms 
are  designed  to  provide  a  firm  guideline  for  the  experiment 
programmer  to  record  all  data  of  interest  for  subseauent 
error  analysis. 

-  Begin  and  end  of  each  steD  was  recorded  with  respect  to 
day  and  time. 

-  Each  error  was  recorded  when  it  is  discovered.  The  error 
was  then  identified  by  a  unique  error  number  (1/2/...). 
Furthermore  the  time  of  discovery  and  the  error  type  (using 
error  tyces  listed  in  ANNEX  F)  were  recorded. 

-  If  aoprooriate/  comments  about  error  discovery/  reason  why 
the  error  was  made/  etc.  were  documented  in  ANNEX  E  2  . 

-  For  each  error  the  phase  in  which  the  error  was  made/  the 
phase  in  which  the  error  was  discovered  and  the  time  spent 
to  correct  the  error  was  recorded  in  ANNEX  El. 

For  each  steo  in  any  one  of  the  software  develooment 
phases  the  day/time  of  begin  and  end  was  recorded.  In  addi- 
tion/ the  time  (in  man  hours)  for  each  step  was  recorded. 
This  excludes  the  overhead  used  for  documentation  of  the 
exoer  i men t  data. 
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ANNEX  A 


Page  1   of  5 


WORKSHEET  FOR  DESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  U 


MAN  !       ! 

STEP 

PROBLEM  AND  PLANNED  SOLUTION 

ALTERNATE 

DAY 

HOURSiERROR!  COMMENT 

n 

/ 

SOLUTIONS 

TIME 

/STEP!   *   ! 

1 

Analyzing  structures  with 

resoect  to  minimum  number  of 

oaths. 

The  minimum  number  of  oaths 

includes  all  oossible  oaths 

starting  at  the  entry  node  (1) 

3/01 
1900 

and  ending  at  any  node  which 

/ 

12. o:     : 

has  no  Successor.    The  minimum 

number  excludes  all  oatis   ha- 

ving reoetative  traversals  of 

1ooos»  such  that  any  seauence 

of  arcs  is  traversed  more  than 

once  i  n  a  row . 

3/03 

2400 

2 

Design  of  Data  Structures: 

3/0a 
1  100 
3/Oa 

1900 

!       [Details 

!      !  see 

!      !  R  e  m  a  r  k  s 

Remarks : 

Two  freelists  are  used  to  sucoly  storaqe  soace  to  keeo  information  about 

oaths   and  the  structure  of  the  unaerlyina  directed  graoh. 


I  terns 
used  for: 


FREELIST  1  (Mi) 
(451  Items  of  U  bytes  each  (32  Bits)) 


flv  t  es 


>  <    2     Bytes 


a) 

Successor 
I  n  forma  t  i  on 
(e 1 emen t 

i  n     forward 
1  i  nlted    list) 


successor  name 


otr  next  successor 


b) 

Header  of 
successor 
1  i  st 

c) 

Free  list 
Header 
(Item  0) 


reachability  oarent  !  otr  successor  list 

• 
• 

0  lotr  next  free  item 
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ANNEX  A 


Page  la  of  5 


Items 
used  for: 


a) 

Node 

In  f  ormat  i  on 

(duD I i cat  e ) 


FREELIST  2  (^2) 
[20001  Items  of  8  Bytes  each  (64  Bits)) 


Bytes 


Bytes 


pointer  to  oriqina1 
ot  r  to  successor 


node  name 

otr  to  predecessor 


0) 

Pat  hheader 
In  format  i  on 


ptr  to  previous  path 


ptr  to  beain  of  oat* 


path  i oent  i  t i ca t  i  on 


ptr  to  next  oat1-! 


c)  ! 

Node 

Node         ! 

Information  Jpointer  to  original 
(original)   ;-------.-----------« 

!  9  of  successors 


noae  name 


otr  successor  list 


d) 

Free' i  st 
Header 
(Item  0) 


otr  next  free  item 
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Typical  usage: 

a)  Original  node  with  successor  list 


i ; 


Pt  r« 


rlotr- 


j  [ptP- 


NIL 


Where: 

i  t     ) i     k  are  node  names 

r  reoresents  reachability  of  node  i 

o  is  the  Storage  location  within  v2 

NIL  inaicates  the  end  of  a  linked  list 

b)  Doubly  linked  list  of  path  headers: 


< ot  r 

!  Ptr 


ot  r- 


■or  r 


ot  r 


i  +1 

otr- 


oegi  nn  i  ng 
of  oath  i 


beg  inning 
of  oath  i  1 1 


Where  i  and  i+1  are  oath  identifications. 

c)  Nodes  (duplicates)  being  elements  of  a  oath 


original  <• 
of  i 


or  i  g i  na  1  <• 
of    J 


---ot  r 
!  ot  r 


i 


Ptr- 

T 


V 
■ot  r 


Ptr 


Pt  r« 


T  oredecessor  of  node  i 


.-/here  i  and  j  are  successive  nodes  on  a  oath. 
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WORKSHEET  FOR  DESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  * 


STEP 


PROBLEM  AND  PLANNED  SOLUTION 


ALTERNATE 
SOLUTIONS 


!  VAN  ! 

DAY  !HOURS!ERROR 

time:/step:     « 


C0MMENT 


TOP  -  DOWN  Design: 
DeHne  the  following  program 
parts: 

-Dat  a  st  rue  t ures 
-Primitives  to  support  data 
St  rue tures 

-Utilities  to  support  data 
St  ruct ures 

-Logical  orocedures  to  support 
analysis  of  directed  graphs 
-Subroutines  to  support  analy- 
sis of  possible  paths   through 
a  d  i  rec t  ed  graph 
-Input  of  data 
-Outout  of  data 
-Error  messages  and  relevant 
diagnostics  to  support  error 
error  investigation 


3/05 
1500 


3/05 
loOO 


1.0 
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WORKSHEET  FOR  DESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  * 


STEP 

n 


PROBLEM  AND  PLANNED  SOLUTION 


ALTERNATE 
SOLUTIONS 


!  MAN  ! 

DAY  IHOURSJERROR 
TIVEI/STEP!  u 


COMMENT 


Define  Data  Structures 

a)  Use  Freel i  st  (Ml )  of  451 

4  byte  w  o  r  a  s  used  to  orovide 
information  about  successors 
of  parent  nodes  (each  of  max. 
30  parent  noses  may  nave  up  to 
14  successors  ) 

b)  Use  Freelist  CM2)  of  20001 
*  byte  items  to  keeo  informa- 
about  original  nodes*  oath 
heaaers  and  oossible  oaths 
through  the  directed  graoh 
bei  ng  ana  1 y :eo. 

(each  item  is  designea  to  hola 
4  integer  values  oetween  0  ana 
&5535) 

Design  of  orimitives  and 
utilities  to  sucoort  aata 
st  rue tures 

a)    Primitives  for  Freelist  1: 
-Initialize  Freelist  1 
-Carl,  Cdrl  (retrieval  of 

i  nt eger  va 1 ues  ) 
-Setcarl/  Setcdrl  (set  values) 
-Allocatel,  Freel  (allocate/ 
f  ree  e 1 ement  s  of  Ml) 


b)  Primitives  for  Freelist  2: 
-Initialize  Freelist  2 
-Car2,  Car2  (retrieval  of 

i nt  eger  values) 
-Setcar2,  Setcar2  (set  values) 
-Allocate2,  Free2  (allocate/ 
free  elements  of  M  2 ) 


3/05 
1600 


3/05 
la30 


3/05 
1630 


3/05 
2030 

3/0o 
1330 


3/06 
1500 


5.5 


see  also 
remarks 
on  oage  1 
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ANNEX  A 


Page  4   of  5 


WORKSHEET  FOR  DESIGN  PHASE  ANO  DESIGN  REVIEW  PHASE  OF  PROJECT  * 


STEP 


PROBLEM  AND  PLANNED  SOLUTION 


ALTERNATE 
SOLUTIONS 


:  man  : 

DAY  IHOURSiERROR 

timei/step:  * 


COMMENT 


Define  Algorithms  'or  Utili- 
ties to  support  structures 
us  i  ng  Ml  : 
-Name*  Setname 
-8rother»  Addbrother 
-Son(i)  (get  name  of  ith  son) 


Def  i 
1 1  es 

us  i  n 

a )  Su 
oat  h 
-Set 
-Lin 
-Ini 
-Nex 

b)  S 
-Nam 
-Pre 
-Sue 
-Num 
-Lin 


ne  A  1  go 
to  suo 

a  M2: 

poort  0 
header 
Path  I 
koath( f 
t  i  al i  ze 
t  oath, 
upoor t 
e  of  ( r 
decesso 
cessor 
ber    of 
king  of 


-Ouo 1 i  ca t  e ( 
-Addnode 
-Ouol i  cat  e 
-Imol ement  a 
oath 


rithms  for  Utili- 
oor t  s t  ructures 

f  linteed  list  of 
s : 

0,  -Path  ID 
orwara/bacUara) 

Path!  i  st 

-Previous  Path 
of  oath  structures: 
etrieve  node  name) 
r 

Successors 
nodes ( forward  and 
bac  xward) 
set  duplicate  node) 

a  Path 

t  i  on  of  alternative 


Define  Procedural  Algorithms: 

-Remove  a  Path 

-List  Path 

-List  all  Paths 

-Fi  nd  End  of  a  Path 

-F  i  na  Original 


3/0o 
1800 


3/0o 
1830 

3/07 

1000 


3/07 
1,200 

3/08 

1500 


1505 
3/08 
1530 


.5 


2.0 


.5 


17 


D9  forme  r 
ana  lysis 
did  not 
i  nc 1 ude 
trivial 
case 
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Page  5   of  5 


WORKSHEET  FOR  DESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  u 


STEP 
* 


PROBLEM  AND  PLANNED  SOLUTION 


ALTERNATE 
SOLUTIONS 


DAY 
TIME 


MAN 
HOURS 
/STEP 


ERROR 
U 


COMMENT 


Des  i  gn  Rev  i  ew  : 
Define  global  oointers: 
-Pat  hhead 
•Currentoat h 
-Last  oat  h 
-Current  node 
-Last  occurence 
Define  algorithm  for  path 
anal ys  i  s  : 
-Set  Paths 

Soecify  logical  algorithms: 
-Occurs  Twice  (node  with  same 
name  on  same  oath) 
-One  way  (loop  without  alter- 
native branches ) 
-Match  (same  sequence  of  nodes 
duplicated  on  current  oath) 
-Checkbac*  (check  for  invalid 
a  1 t ernat  i  ves ) 


3/09 

1100 


3.0 


3/09 
1400 
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ANNEX  C 


Page  1  of  2 


WORKSHEET  FOR  CODING  PHASE  OF  PROJECT  * 
Beginninq  of  Coding  (day/time)  :  3/06/1500 
End  of  Coding    (day/time)  :  3/10/1730 
Man  hours  :  26  (including  punching  of  cards) 


CODING 


BEGIN 
DAY/TIME 


END 
DAY/TIME 


PROGRAM  PART 


1) 

EPPORIDAY 
«   [TIME 


COMMENT 


03/06/1500 

03/06/1630 

03/06/1830 
03/07/1200 

03/08/1000 


03/06/to30 

03/06/1800 
03/06/2030 

03/07/1400 


03/08/1530 

03/08/1900 
03/09/2000 


03/08/1200 

03/08/1800 
03/08/2000 

03/08/2200 


•Data    Definition 
■P  r  i  m  i  t  i  ves 
'Error    Handling 
•Initialization 

•Punch  i  ng    cards 


•Utilities 
•Utilities 


•Punch  i  ng    cards 
(Ut  i  1 i  t  i  es  ) 


■Codi  ng    of    or  o- 
cedural     suorou- 
t  i  nes 

•Coding    of    pro- 
cedural    subrou- 
t  i  nes 

•Punch  i  ng    cards 


3/06! 

:     i 

1530! 

:     2 

1540! 

:     3 

16201 

!   4 

1740! 

3/07! 

!   5 

1305! 

3/08! 

:   io 

1000! 

:   i  i 

1005! 

!  12 

1005! 

:   13 

1010! 

:   iu 

1025! 

:   15 

1030! 

:   16 

1030! 

!  18 

1630! 

19 

2130! 

20 

2150! 

I)     Record    when    error     is 
detected . 

C28 
C28 
C23 


C28 


(nissing  BEGIN) 
( m  i  spe 11  i  ng  of 
procedure  name) 


D9 


C12 
C28 

C23 

C16 

CI 

C17 

C17 


09  (design  did  not 
consider  removal  of 
first  and  last  oath) 


Dl  1  ( f au) ty  design 
of  index  calculation) 

C10  (faulty  condition 
"Not  Equal"  instead  of 
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ANNEX  C 
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WORKSHEET  FOR  COOING  PHASE  OF  PROJECT  u 
Beginning  of  Coding  (day/time)  :  3/06/1500 
End  of  Coding    (day/time)  :  3/10/1730 
Man  hours  :  26  (including  punching  of  cards) 


COOING 
8EGIN      !     END 
DAY/TIME   !  DAY/TIME 


PROGRAM  PART 


1) 

ERRORJOAY 
*   JTIME 


COMMENT 


03/09/1200 
03/09/1300 
03/09/1500 

03/09/2000 
03/09/2100 

03/10/1000 
03/10/1 100 

03/10/1630 


03/09/1300 
03/09/1500 

03/09/1730 
03/09/2100 

03/09/2300 

03/10/1  100 
03/10/1200 
03/10/1730 


■Codi  ng  o  f  I/O 
subrou t  i  nes 

■Punc  h i  ng  cards 


■Cod i  ng  of  Log i ■ 
cal  orocedures 


-Punching  cards 


■Cod  i  ng  of  path 
anal y 2  i ng  a  1  go* 
r i  thm( "Set 
oaths" ) 

■Punching  caras 


-Coding  of  I/O 
suo rout  i  nes 

-Punch  inq  cards 


3" 


35 
36 


40 


12 


U4 


U6 


3/09 


luao 


1530 
1615 


2030 


2200 


3/10 
10U0 


1700 


1)  Record  when  error  is 
det  ec t  ed  . 


CI  I 


09 
09 


C20 


09 


015 


09 
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WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  n    :    2  DEBUG  Run  #  :  1 

Begin  of  Debug  Run  (day/time)  :   03/06/1800 

End  of  Debug  Run    (day/time)  :   03/07/iaOO 

*  of  Debug  Steps  incl.  in   Debuq  Run:  «    CPU  time  for  Debug  run  (sec):  1.82 

CPU  time  for  necessary   compiles  (sec)  :  4.33 

a)   0.89    b)   1.76    c)   1.68    a)         e)  f)         q) 

Man  hours  for  this  Debug  Run  :  H . 0  (including  preparation  of  debug  run) 

MAN        1) 


STEP 
a 


PROGRAM 
PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY  JHOURSIERROR 
TIMEJ/STEP!   * 


COMMENTS 
AND  CODED 
ERROR  TYPES 


Pr  i  m  i  - 
t  i  ves 
and 

part  of 
ut  i  1  i  - 
t  i  es 


Pr i  mi  > 
t  i  ves 


Get  error  free  comoile 


Repeat  steo  1 

Check  i n i t i a  1  i zat i on , 
allocation  and  freeing  of 
items  in  ooth  freelists. 
Check  oarameter  implemen- 
tation. 

Check  writing  of  blank 
1  i  nes  . 

(pointers  of  freelists 
must  be  set  aporopr i a t e 1 y , 
allocated  elements  must  be 
f  i 1 1 ed  with  zeroes  » 
allocated  items  iust  be 
un 1  i  nkea» 

after  freeing  an  element 
becomes  oart  of  the  free- 
list  agai  n, 

parameters  should  be  set 
as  Designed) 
reoeat  steos  1  and  3 


3  comoi 1 e 
errors 


O.K. 


3/06 
181  1 


3/06 
1<»30 

3/07 
1  1  10 


uooer 
half  of 
a  1 1 ocat  ed 
el ement 
of  M2  not 
f i 1  led 
with 
zeroes 


O.K. 


1250 


3/07 
U00 


1.3 

1.2 

0.5 


1  .0 


1  )  Record  when 
error 
occurs 

Al 
C23 

At 


A  1   ("  +  1  " 

left  out 

while  pun- 
ching cards 
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ANNEX  D 


Page  2  of  11 


WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  «  :  2  DEBUG  Run  *  :  2 

Begin  of  Debug  Run  (day/time)  :   03/08/1000 

End  of  Debug  Run    (day/time)  :   03/08/1700 

*  of  Debug  Steps  incl.  in   Debug  Run:   2    CPU  time  for'  Debug  run  (sec):  .46 

CPU  time  for  necessary   comoiles  (sec)  :  2.23 

a)   2.23    b)         c)         a)        e)  f)        g) 

Wan    Hours  for  this  Debug  Run  :  2.5  (including  oreoaration  of  deouq  run) 

MAN         1) 


STEP! PROGRAM 

OBJECTIVE  AND    EXPECTED 

ACTUAL 

DAY 

HOURS  iERROR 

COMMENTS 

a  !   PART 

RESULT 

RESULT 

TIME! /step:   « 

t 

AND  CODED 
ERROR  TYPES 

1)  Record  inen 

e  r  r  c  r 

OCCurs 

1   !  Pr i  mi  - 

Check  naxiTum  number  of 

3/08 

! t  i  ves 

allocations  in  both  free- 

1000 

2  man  hours 

!  and 

1 i  st  s  . 

0.3  : 

scent  in 

!  oar t  of 

(all  elements  exceot  free- 

O.K. 

oreoarat  ion 

! ut  i  1  i  - 

list  headers  are    alloca- 

of deouq 

!  t  i  es 

ted) 

run 

2   !  E r ror 

Check  error  messages  for 

!  hand" 

exhaustion  of  each  free- 

3/08 

0.2  ! 

'ling 

list. 

O.K. 

1700 
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ANNEX  D 


Page  3  of  11 


WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  tf    :    2  DEBUG  Run  *  :  3 

Begin  of  Debug  Run  (day/time)  :   03/08/2200 

End  of  Debug  Run    (day/time)  :   03/09/1300 

0  of  Debug  Steps  inel.  in   Debug  Run:   7    CPU  time. for  Debug  run  (sec):  .6 

CPU  time  for  necessary   comDiles  (sec)  :  18.12 

a)   2.67    b)   ^.75    c)   2.75    a)  2.U3    e)  3.U9     f)  a. 03    g) 

Man  hours  for  this  Cecug  Run  :  u.5  (including  oreoaration  of  debug  i*un) 

MAN        1) 


STEP 


PROGRAM 
PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY  !  HOURS  JERROR 
TIMES/STEP |   a 


COMMENTS 
AND  CODED 
EPROR  TYPES 


P  r  i  m  i  - 
t  i  ves» 
Error 

hand* 
1  i  ng, 
Proce- 
dural 
suOrou- 
t  i  nes 


Get  error  'res  comoile  of 
all  subroutines  and  ori- 
mitives  excludina  I/O  s  u  o  ■ 
routines  ana  logical  sub- 
rout  i  nes  . 


ReDeat  steo  1 

Reoeat  steo  1 

Reoeat  steo  1 

Reoeat  steo  1 


Reoeat  steo  1 

Reoeat  debug  runs  1  and  2 
(same  results  exoected) 


75  com- 
oile d  i  a- 
gnost  i  cs 
( see 

remarks) 


O.K. 
O.K. 


!  3/08! 

12200! 

12200! 

!22«5 

12310! 

12323! 

,'3/09! 

!  1030! 

!  1130 j 

! 1150! 

!3/09j 

1  .0 


0.5 

0.5 
0.5 
1  .0 


0.3 
1  .2 


Remarks : 

The  ALGOL  comoiler  diagnostics  do  not  always  allow  a  comolete 

investigation  of  all  errors. 


21 
22 
23 
2a 


25 

26 

27 

28 
29 
30 
31 
32 
33 


1  )  Record  when 
error 
occurs 


C23 
CI  7 

C27 

Al 


C17 

A2 

C28 

Al 
Al 
Al 
Al 
Al 
C27 
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ANNEX  0 
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WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  »    :  2  OEBUG  Run  «  :  a 

Begin  of  Debug  Run  (day/time)  :   03/09/1500 

End  of  Debug  Run    (day/time)  :   03/10/1200 

*  of  Debug  Steps  incl.  in   Debug  Run:   6    CPU  time  for  Debug  run  (sec):  .61 

CPU  time  for  necessary   comoiles  (sec)  :  23.45 

a)   3.67    b)   5.02    c)   5.20    a)  a. 60    e)  4.96     f)         g) 

Man  hours  for  this  Debug  Run  :  3.0  (including  oreoaration  of  debug  run) 

MAN        1) 


STEPJPROGRAM 
«  '   PART 


OBJECTIVE  AND    EXPECTED  !   ACTUAL 
RESULT  !   RESULT 


DAY IHOURSiERROR 

time:/step:  u 


COMMENTS 
AND  CODED 
ERROR  TYPES 


A)  1 

subro 

t  i  nes 

exceo 

1  ogi  c 

proce 

dures 

I/O 


Remarks:  F 
c 


Get  error  free  comoile. 


Repeat  steo 
Check  input 
cases 


1 
for 


trivial 


Repeat  steo  3 
and  preceai  ng 
t  i  on  of  f  ree lists  as 
as  initialization  of 
of  path  headers . 
Check  function  "SON" 
correct  return  values 
(trace  should  match  * i t h 
design  considerations) 


ReDeat  steo  a 


with  trace 
initial i  za- 
wel  1 
1  i  st 

for 


Repeat  step  4 


unctions  and  subroutines  are 
ases  and  Doundary  conditions. 


2  como  i 1 e 
errors 


O.K. 
numbe r  of 
succes- 
sors d  1  a- 
ced  in 
wrong 
f  i  e  1  a 


w  r ong  v  a- 
1 ue  re- 
t  urned 
f  r om  "Num 
ber  of 
Succes- 
sors" 
Mi  s s  i  n g 
1  ink 

O.K. 


3/10 
1100! 

first  checked  for 


3/09 
1500 

1730 


17  30 
1950 

200C 


2130 


2200 


2320 
2340 
2350 


0.5 


0.5 


0.5 


0.5 
0.5 


37 
38 


39 


ai 


43 


1  )  Record  when 
error 
occurs 


C23 
Al 


C26  (READ  and 
READON  con- 
fused) 


Al 


C9 


trivial 
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ANNEX  0 
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WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  n    :  2  DEBUG  Run  u    :  5 

Begin  of  Debug  Run  (day/time)  :   03/10/1600 

End  of  Debug  Run    (day/time)  :   03/10/2*400 

*  of  Debug  Steps  incl.  in   Debug  Run:   5    CPU  time  for  Debug  run  (sec):  1.35 

CPU  time  for  necessary   compiles  (sec)  :  29. 49 

a)   5.71    o)   5.94    c)   5.43    a)  5.79    e)  6.62     f)         g) 

Man  hours  for  this  Debug  Run  :  5.0  (including  preparation  of  debug  run) 

MAN        1  ) 


STEPiPROGRAM 
8  I   PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY  jHOURS IERR0R 

timej/step:  3 


COMMENTS 
AM0  CODED 
ERROR  TYPES 


1 

Al  ] 

Pr  i  m  i 

t  i  ves 

and 

ut  i  1  i 

t  i  es 

2 

3 

Check  for  trivial  inouts 
for  all  primitives  ana 
utilities. 


Repeat  steo  1 
Repeat  steo  1 


Repeat  step  I 


Repeat  steo  1 


1  error 
in  link" 
i  ng 


1  error 

3  oroar am 

errors 

found 


2  program 
errors 


O.K. 


3/10 
1530 

1600 


1730 

1930 
1930 


2100 


2130 


2200 
2250 


2300 

3/10 

2330 


0.5 


1  .0 
2.0 


1  .0 


0.5 


45 


47 


48 


49 


50 


51 


52 


1  )  Record  when 
error 
occurs 


C27 


D9 


C21  (state- 
ments in  re- 
verse order ) 
D  1  2  (wrong 
St  oopi  ng 
condition) 
C8  (faulty 
initial ization 
of  I  OOP) 


B4  (error 
in  correct' 
i  ng). 
C24 


Remarks:  Procedures  checked  for  trivial  cases  and  boundary  conditions. 
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WORKSHEET  FOR  OEBUGGING  PHASE 

PROJECT  ft  :  2  DEBUG  Run  ft  :  6 

Begin  of  Debug  Run  (aay/tirne)  :   03/10/2230 

End  of  Debug  Run    (day/time)  :   03/11/1700 

ft  of  Oebug  Steos  incl.  in   Debug  Run:   5    CPU  time  for  Debug  run  (sec):  2.14 

CPU  time  for  necessary   comoiles  (sec)  :  26.3 

a)   5.14    b)   6.62    c)   7.38    a)  7.16    e)  f)         g) 

Van    hours  for  this  Debug  °  u  n  :  6.0  (including  preparation  of  debug  run) 

MAM         1) 


STEPJPROGRAM 

ft  !   PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY  J  HOURS  JERROR 
TIme;/step:   3 


COMMENTS 
AMO  CODED 
ERROR  T/PES 


Al  1 


Loai  ca 1 
proce- 
dures 
and 

wr  i  t  i  ng 
of  a 
oath 


Get  error  free  compile  of 
all  program  oarts. 


Reoeat  step  1 


Check  for  trivial  cases 
while  analyzing  one  oath. 
Check  writing  of  a  path, 
(all  functions  shoula  oer- 
f orm  as  des  i  aned ) 


Repeat  steo  3 


Repeat  steo  3 


8  como i 1 e 
errors 


O.K. 


1  error 
found 


2  errors 
found 


O.K. 


3/10 
2300 
2310 


2330 
3/10 
2400 

3/11 

1400 
1400 


1545 
1620 


1630 

1630 
3/11 

1700 


1  .0 


0.5 


3.0 


1  .0 


0.5 


1  ) 

Record  when 

error 

occurs 

53 

Al 

54 

Al 

55 

Al 

5e 

Al 

57 

Al 

58 

C26 

59 

C9 

60 

C9 

61 


62 


63 


C27  (reset 
forgot  t  en ) 


C10  ( w  rong 
stopping  con- 
di  t  i  on  ) 
C30 
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WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  *    :  2  DEBUG  Run  *  :  7 

Begin  of  Debug  Run  (day/time)  :   03/11/1700 
End  of  Debug  Run    (day/time)  :   03/12/1600 

*  of  Debug  Steps  incl.  in   Debug  Run:   5    CPU  time  for  Debug  run  (sec):  2.61 
CPU  time  for  necessary   comoiles  (sec)  :  38.78 
a)   8.05    b)   7.81    c)   7.88    e)  7.82    e)  7.22     f)         g) 


Man  hours  for  this  Debug  Run 


7.0  (including  preparation  of  debug  run) 

MAN        1) 


STEP 

PROGRAM 

OBJECTIVE  AND    EXPECTED 

ACTUAL 

!  DAr 

HOURS 

EPROR 

!  COMMENTS 

* 

PART 

RESULT 

RESULT 

:  time 

'/STEP 

1 

!  AND  CODED 
!  EPROR  TYPES 

1)  Record  »nen 

error 

1 

Proce- 

Check analysis  of  oatnes. 

2  proaram 

3/1  1 

0.5 

occurs 

dure 

(All  pathes  are     implemen- 

errors 

1700 

"Set 

ted  indoubly  linked  lists) 

1730 

64 

D  1  2  (wrong 

oaths" 

1745 

65 

s  t  oop  i  ng  con- 
dition) 
D  1  I  (  w rong 
i  noex  i  ng ) 

2 

Reoeat  steo  1 

1  error 

1805 

0.5 

found 

1810 

66 

011  ( same  as 
error  65 ) 

3 

Reoeat  steo  1 

5  errors 
found 

1820 

3/12 

1300 
1315 
1320 

1325 
1340 

3.5 

67 

68 
69 

70 
71 

D3  ( forgot  ten 
03   uoaate 
03  of  a  vari- 
able) 
C2<9 

B4  (misinter- 
pretation of 
error  source) 

4 

Repeat  steo  1 

1  error 
f  ouna 

1400 

1.5 

72 

D9 

5 

Reoeat  steo  1 

O.K. 

1530 
3/12 

1600 

0.5 
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WORKSHEET  FOR  OEBUGGING  PHASE 

PROJECT  n    :  2  DEBUG  Run  *  :  8 

Begin  of  Debug  Run  (day/time)  :   03/13/1600 

End  of  Debug  Run    (day/time)  :   03/l«/l600 

*  of  Debug  Steos  i nc 1 .  in   Debug  Run:   3    CPU  time  for  Debug  run  (sec):  162.75 

CPU  time  for  necessary   comoiles  (sec)  :  19.59 

a)   6.63    b)   6.U5    c)   6.51    a)         e)  f)         g) 

Man  hours  for  this  Debug  Run  :  17.0  (including  Dreoaration  of  aebug  run) 

VAN        1  j 


STEPJPROGPAM!  OBJECTIVE  AND    EXPECTED 

ACTUAL 

DAY 

HOURS 

ERROR 

COMMENTS 

tt     !   PART  !           RESULT 

RESULT 

TIME 

/STEP 

a 

AMD  CODED 
ERROR  TYPES 

1)  Record  when 

error 

occurs 

1   {All     ! Check  various  small  direc- 

1  error 

3/13 

!        Ited  qraphs  for  boundary 

f  ouno 

1600 

8.0 

!        ! conoi  t  i  ons  . 

in  -Mat- 

ching" 

2000 

73 

C2°-  (wrong 

oranching) 

2   !        IReoeat  steo  1 

1  error 
found 

3/l« 
0  70  0 

6.0 

(  some 

1  too 

7a 

010 

oaths 

sk  i  oped 

du  r  i  ng 

analysis) 

3   !        JReoeat  steo  1 

O.K. 

1300 
3/14 

1600 

3.0 

Remarks:  Error  analysis  during  this  deoug  run  was  extremely 

difficult  and  time  consuming  for    the  following  reasons: 

-  Most  results  were  verified  by  desk  checking. 

-  The  i mo  1 emen t at i on  of  oaths  was  checked  by 
examining  the  contents  of  allocated  items  of 
both  f reel i  sts  (Ml ,M2)  . 

-  Some  structures  being  tested  had  a  very  large 
number  of  oaths  (  >  2000). 

-  The  orogram  contains  a  great  number  of  rather  complex 
algorithms*  each  of  which  had  to  be  tested  with 
resoect  to  its  boundary  conditions. 
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WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  »     :     2  DEBUG  Run  U     :     9 

Begin  of  Debug  Run  (day/time)  :   03/13/1100 

End  of  Debug  Run    (day/time)  :   03/15/2300 

«  of  Debug  Steps  incl.  in   Debug  Run:   1    CPU  time  for  Debug  run  (sec):  3.92 

CPU  time  for  necessary   compiles  (sec)  :  7.52 

a)   7.52    b)  c)  c)         e)  f)         g) 

Man  hours  for  this  Debug  Run  :  3.0  (including  preparation  of  deouq  run) 

MAN        1  ) 


STEP 
a 


PROGRAM 
PART 


OBJECTIVE  AMD    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY [HOURS ! ERROR 

time;/step:  a 


C0M"EMT5 
Af:0  CODED 
ERROR  TYPES 


Al  I 


Check  internal  error 

messages  . 

(Error  messages  5,o,7, 

11»12,13  ShoulO  be  orin> 

ted) 


O.K 


3/1" 

2545 


3/15 
2300 


3.0 


1)  Recor-1  **en 
error 
occurs 
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WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  *  :  2  DEBUG  Run  *  :  10 

Begin  of  Debug  Run  (day/time)  :   03/16/1500 

End  of  Debug  Run    (day/time)  :   03/16/1600 

*  of  Debug  Steps  rncl.  in   Debug  Run:  1     CPU  time  for  Debug  run  (sec):  6.75 

CPU  time  for  necessary   comoiles  (sec)  :  7.73 

a)   7.73    b)  c)  o)         e)  f)         g) 

Man  hours  for  this  Debug  "un  :  1.0  (including  orecaration  of  debug  run) 

MAN        i  ) 


STEP 


PROGRAM 
PART 


OBJECTIVE  AMD    EXPECTED 
RESULT 


ACTUAL 
RESULT 


OAY  J^OURS ! EPROR 
TIMEJ/STEP!   3 


COMMENTS 
AND  CODED 
ERROR  TYPES 


Al  1 


Chec«  some  user  deoenaent 
error  messages . 
(error    messages  10  ana  8 
should  be  orinted) 


O.K. 


3/1*3 

150  0 


3/16 

loOO 


t  .0 


1)  Record  when 
error 
occurs 
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WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  U    :  2  DEBUG  Run  »    :  1  1 

Begin  of  Debug  Run  (day/time)  :   03/16/1600 

End  of  Debug  Run    (day/time)  :   03/17/1200 

*  of  Debug  Steps  inc).  in   Debug  Run:  2     CPU  time  for  Debug  run  (sec):  6.45 

CPU  time  for  necessary   comoiles  (sec)  :  14.95 

a)   7.90    b)   7.05    c)  o)         e)  f)         g) 

Man  hours  for  tMs  Debug  Run  :  2.0  (including  creoaration  of  oeOuq  run) 

MAN        1) 


STEP 

4 


PROGRAM 
PART 


OBJECTIVE  AMD    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY ! HOURS :error 
timei/step:  a 


COMMENTS 
AND  CODED 
ERROR  TYPES 


Al  I 


Chec'e  error  message  *  R 


Reoeat  steo  1 


error 
messaae 
s  R  not 
or  i  nt  ed 


O.K. 


3/16 

1&00 
1800 


3/17 
1  140 
3/17 

1200 


1  .0 


1  .0 


75 


1)    Record    *rier\ 
error 
occurs 


C21 
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ERROR  LISTING 
PROJECT  U    :  2 

Begin  of  Project  (day/time)  :  03/01/1900 
End  of  Project  (day/time)  :  03/17/2200 
Man  hours  for  total  project  :  125.0 


ERROR 

PHASE 

!   PHASE 

!      ERROR 

!    TIME 

!    *  of  OTHER 

U 

in  which 

in  which 

,      TYPE 

. soent  t  o 

STATEMENTS  OR 

ERROR  was 

ERROR  was 

(see  ANNEX  F) 

[solve  the 

,  PARTS  OF  THE 

di  s- 

made 

!  ERROR 

PROGRAM 

covered 

(Man 

min.) 

AFFECTED 

1 

Coding 

Codi  ng 

C28 

!     5 

2 

Cod  i  ng 

Codi  ng 

C28 

5 

3 

Coding 

Coai  ng 

C23 

2 

a 

Coa  i  ng 

Coa  i  ng 

C28 

1 

5 

Debuggi  nq 

Coai  ng 

Al 

5 

6 

Oebuggi  ng 

Cod  i  ng 

C23 

5 

7 

Debuggi  ng 

Coai  ng 

Al 

5 

3 

Oebuggi  ng 

Cod  i  ng 

Al 

10 

0 

Codi  ng 

Oes  i  gn 

09 

15 

flho 1 e  a  1  go  r i  t  hm 

affected. 

10 

Codi  ng 

Codi ng 

C12 

1 

1  1 

Codi  ng 

Coding 

C2% 

1 

12 

Codi  ng 

Codi  ng 

C23 

1 

13 

Coa  i  ng 

Codi  ng 

C16 

1 

l« 

Codi  ng 

Coai  ng 

CI 

5 

15 

Coai  ng 

Coding 

C17 

2 

16 

Cod  i  ng 

Cod  i  ng 

C17 

2 

17 

Oes  i  an 

Des  i  gn 

09 

1 

18 

Codi  ng 

Des  i  gn 

09 

30 

whole  a  1  go  r  i  t  hm 

af  f ec  t  ed. 

19 

Codi  ng 

Des  i  gn 

Oil 

5 

20 

Codi  ng 

Codi  ng 

CIO 

5 

21 

D»buggi  ng 

Cod  i  ng 

C23 

5 

22 

Oebuggi  ng 

Cod i  ng 

C17 

15 

23 

Oebugg  i  ng 

Coding 

C27 

5 

2a 

Oebugg  i  ng 

Codi  ng 

Al 

2 

25 

Oebuggi  ng 

Coding 

C17 

5 

26 

Debugai  ng 

Codi  ng 

A2 

10 

27 

Debuggi  ng 

Codi  ng 

C28 

10 

28 

Debuggi  ng 

Codi  ng 

C17 

5 

29 

Oebuggi  ng 

Cod i  ng 

Cll 

5 

30 

Debuggi  ng 

Codi  ng 

Al 

15 
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ERROR  LISTING 
PROJECT  »    :  2 

Begin  of  Project  (day/time)  :  03/01/1900 
End  of  Project  (aay/time)  :  03/17/2200 
Man  hours  for  total  project  :  125.0 


ERROR 

PHASE 

PHASE 

ERROR 

TIME 

,    *  of  OTHER 

n 

in  which 

in  which 

TYPE 

soenf  to 

STATEMENTS  OR 

ERROR  was 

ERROR  was 

(see  ANNEX  F) 

solve  the 

PARTS  OF  THE 

ai  s- 

made 

ERROR 

PROGRAM 

covered 

(Man 

utn.) 

AFFECTED 

31 

Debuaai  ng 

Coding 

Al 

5 

32 

Oebuggi  ng 

Coa  i  ng 

M 

2 

33 

Debuggi  ng 

Codi  ng 

C27 

5 

34 

Coai  ng 

Cod  i  nq 

CI  1 

1 

whole  algorithm  affected. 

35 

Codi  ng 

Des  i  gn 

09 

15 

whole  algorithm  affected. 

36 

Codi  ng 

Des  i  gn 

09 

15 

37 

Oebuggi  ng 

Codi  ng 

C23 

5 

38 

DeDugg  i  ng 

Coa  i  ng 

Al 

5 

39 

Deouggi  ng 

Cod i  ng 

C26 

30 

uO 

Debuggi  na 

Cod  i  ng 

C20 

5 

41 

Debugg  i  ng 

Cod  i  ng 

Al 

5 

42 

Codi  ng 

Des  i  gn 

09 

20 

whole  algorithm  affected. 

43 

Deouggi  ng 

Des  i  gn 

09 

10 

44 

Codi  ng 

Des  i  gn 

015 

5 

45 

Debuggi  ng 

Coai  ng 

C27 

10 

46 

Codi  ng 

Desi  gn 

09 

5 

47 

Debuggi  ng 

Desi  an 

09 

60 

Whole  subroutine  affected. 
(3  changes  necessary) 

48 

Debuggi  ng 

Cod i  ng 

C21 

10 

49 

Deougai  ng 

Des  i  gn 

D12 

5 

50 

Debuggi  ng 

Cod  i  ng 

C8 

15 

51 

Debugai  ng 

Debugai  ng 

B4 

20 

52 

Debuggi  ng 

Cod  i  ng 

C24 

5 

53 

Debugg  i  ng 

Cod  i  ng 

Al 

5 

54 

Debuggi  ng 

Codi  ng 

Al 

2 

55 

Deouggi  ng 

Codi ng 

Al 

2 

56 

Debuggi  ng 

Codi  ng 

Al 

2 

57 

Debuggi  ng 

Codi  ng 

Al 

2 

58 

Debugg  i  ng 

Codi  ng 

C28 

5 

59 

Debuggi  ng 

Codi  ng 

C9 

5 

60 

Debuggi  ng 

Codi  ng 

C9 

5 
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ERROR  LISTING 
PROJECT  u    :  2 

Begin  of  Project  (aay/time)  :  03/01/1900 
End  of  Project  (day/time)  :  03/17/2200 
Man  hours  for  total  project  :  125.0 


ERROR 

PHASE 

PHASE 

ERP0R 

TIME 

n    of  OTHER 

it 

in  which 

in  which 

TYPE 

spent  to 

STATEMENTS  CR 

ERROR  was 

ERROR  was 

(Se(?  ANNEX  F) 

solve  the 

PARTS  OF  THE 

oi  s- 

made 

ERROR 

PROGRAM 

covered 

(^an 

■ntn.) 

AFFECTED 

ol 

Debugo  i  ng 

Cod  i  ng 

C27 

30 

62 

Debuggi  ng 

Codi  ng 

C10 

10 

63 

Debuggi  ng 

Coding 

C29 

10 

&a 

Debuggi  ng 

Des  i  gn 

D12 

10 

65 

Debuggi  ng 

Des  i  gn 

Dll 

5 

66 

Oebuggi  ng 

Des  i  gn 

01  1 

15 

67 

Debugqi  ng 

Des  i  gn 

03 

10 

68 

Debuggi  ng 

Des  i  on 

03 

10 

69 

Debugqi  ng 

Des  i  gn 

03 

5 

70 

Deouggi  ng 

Codi  ng 

C30 

15 

71 

Debuggi  ng 

Debugqi  ng 

6a 

10 

72 

Debugai  ng 

Des  i  gn 

D9 

120 

73 

DeDugai  ng 

Cod  i  ng 

C29 

180 

7a 

Debuggi  ng 

Des  i  gn 

D10 

60 

75 

Debuggi  ng 

Codi  ng 

C21 

30 
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ERROR  LISTING  (COMMENTS) 


ERROR 
u 


DAY 
TIME 


COMMENTS 

(EVIDENCE,  THOUGHTS,  WHY  WAS  THE  ERROR  MADE? 

WHY  AND  HOW  WAS  THE  ERROR  DISCOVERED? 

ERROR  BLOCKING,  etc.) 


03/06 

1 

1530 

2 

15«0 

3 

1620 

a 

17U0 

5 

1815 

b 

1815 

7 

1815 

03/07 

8 

1305 

9 

1305 

03/08 

10 

1000 

u   : 

1005 

12 

1005 

13   ! 

1010 

la   ! 

1025 

15   ! 

1030 

16   ! 

1030 

17  : 

1505 

IB   , 

1630 

19 

2130 

20 

2150 

21   ! 

2200 

22 

2200 

23 

2200 

2a 

2200 

25 

22U5 

26 

2310 

27 

2323 

03/09 

28 

1030 

29 

1030 

30 

1030 

31 

1030 

32 

1030 

Errors  1,2,3  were  discovered  while  reading  previously 
written  sections  of  code. 


Lac*  of  concentration  while  punching  cards. 

(Main  disadvantage  while  Punching  cards  is  that  ounched 

data  is  not  immediately  seen  after  each  key  strode.) 

same  as  a 

Programmer  did  not  cneck  programming  manual,  (error  could 

have  been  avoided) 

same  as  U 


Errors  10-17  were  detected  Because  oroarammer  ounched  caras 
himself.  It  is  Quite  natural  that  he  uses  this  time  to  review 
h  i  s  code . 


Error  found  during  desk  test. 
Error  found  while  punching  caras. 
same  as  19 

Function  name  used  as  local  variable. 

Incomplete  correction  of  error  a  22. 

Mandatory  declaration  omitted  while  punching  cards 

Right  parentesis  omitted. 


Lack  of  concentration  whi 
(Programmer  was  tired.) 
same  as  28 
same  as  28 
same  as  28 
same  as  28 


e  punch  i  ng  caras . 
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ERROR  LISTING  (COMMENTS) 


ERROR 

n 


DAY 
TIME 


COMVENTS 

(EVIDENCE,  THOUGHTS,  WHY  WAS  THE  ERROR  MADE? 

WHY  AND  HOW  WAS  THE  ERROR  DISCOVERED? 

ERROR  SLOCKING,  etc.) 


03/09 

33 

1030 

3a 

1440 

35 

1530 

36 

1615 

37 

1730 

38 

1730 

39 

2000 

no 

2030 

41 

2200 

42 

2230 

U3 

2340 

03/09 

aa 

1040 

03/10 

45 

1600 

a& 

1700 

47 

1730 

48 

1930 

49 

2100 

50 

2130 

51 

2250 

52 

2300 

53 

2310 

54 

2310 

55 

2310 

56 

2310 

57 

2310 

58 

2310 

59 

2310 

60 

2310 

03/11 

61 

1400 

62 

1620 

63 

1630 

64   ! 

1730 

65 

1745 

66    | 

1810 

Extreme  conditions  not  carefully  analyzed. 

same  as  35 

Lack  of  concentration  while  coding. 

"REAO"  and  "READ0N"  confused. 

Using  a  local  variaole  name  from  different  subroutine. 


Use  of  wronq  oarameter. 

Error  found  while  punching  cards. 


Thinking  of  boundary  conditions  while  Duncriing  cards. 
Des*  test  was  not  made  carefully  enough,  (one  case 
omi  1 1 eo) 
Two  statements  in  reverse  order. 

Error    was  discoverea  during  Structured  .Valk  Through. 
Caused  by  changing  code. 

Lack  of  concentration  while  ounching  cards. 

(see  comment  *  4) 

same  as  53 

same  as  53 

same  as  53 

same  as  53 

Programmer  did  not  check  orevious  procedure  declaration, 
( f au I t  y  i  nvocat  i  on ) 
same  as  59 

Forgotten  reset  of  parameter. 

Inappropriate  stopping  condition  in  loop. 

(confused  with  similar  condition  in  a  previuos  problem) 

Wrong  use  of  a  subroutine. 

Error  found  using  a  trace. 

same  as  64 

Error  founa  during  desk  test. 
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ERROR  LISTING  (COMMENTS) 


ERROR 

DAY 

COMMENTS 

4 

TIME 

(EVIDENCE,  THOUGHTS,  aHY  WAS  THE  EPROR  "<>OE? 
WHY  ANO  HOW  WAS  THE  ERROR  DISCOVERED? 
ERROR  BLOCKING,  etc.) 

03/12 

67 

1300 

Necessary  update  of  a  variaole  left  out. 

68 

1315 

same  as  b7 

69 

1320 

same  as  b7 

70 

1325 

6oolean  expression  does  not  meet  the  needs  of  the 
algorithm.  Error  found  by  tracing. 

71 

13U0 

M i s i n t erp r t at i on  of  the  error  source.  Error  founa 
while  repeating  debugaing  steo. 

72 

laOO 
03/ia 

Different  behaviour  of  algoritm  for  removal  of 

last  path  previous  to  the  last  one  *as  not  consiaered. 

Error  founa  Ov  des*  test  of  all  relevant  subroutines. 

73 

2000 
03/15 

Error  founa  during  desk  test. 

7a 

1  100 
03/16 

75 

1800 

Error  found  during  desk  test. 
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PROJECT  *   2 

FINAL  STATISTICS 

Project  name  :  PATH  ANALYSIS  IN  DIRECTED  GRAPHS 

Short  descriction: 

This  proQram       is  designed  to  find  all  oossible  oaths  of 
minimal  length  (i.e.  if  a  directed  qraoh  contains   looos   no 
looo   may   be   traversed   more  than  once  in  a  row)  through  a 
di  rec  ted  gr aoh . 
Inout:  via  ounch  cares  Dugout:  via  line  orinter 

Quantitative  measures: 

1.  *  of  source  statements  :  712 

2.  Total  man  hours  for  oroject  :  125.0 

3.  yan  hours  scent  in 

a)  Design  :  31.0 

b)  Coding  :  26.0 

c)  Debugging  :  55.0 

d)  Testing  :  13.0 

4 .  CPU  time  for  comoiles:  206.65  sec. 

5.  CPU  time  for  debug  runs:  139.46  sec. 
o.  CPU  time  for  test  runs:  8t>9.73  sec. 

7.  a  of  test  and  debug  runs:  13 

8.  «  of  test  and  deoug  steps:  43 

9.  a  of  errors  found:  75 

10.  Total  man  hours  used  to  correct  errors:  16.5 
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T  #  of  errors 


45 


uo 


35 


30 


25 


20 


15 


10 


11.  Error  Detection  : 

Mean  time  between  error  detections:  69  man  min. 


1 

(measured 


*  *** 

*  **  *    * 

*  ***  *    *  *****  *** 

-+•  +  -+-'!'-  +  ._+-  +  -  +  -  +  -*-  +  -+-+-  +  -  +  ..  +  ..  +  -  +  -  +  -  +  -+•-+..  +  -  + 

2         3         a         5         b         7         8         9101112       13 
TIME    BETWEEN    ERROR    DETECTIONS 
from    ooint     in    time    of    detection    of    previous    error) 


man    hours 
♦-+-->    of 

oro  j  ec  t    time 
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12.  Error  Correction: 

Mean  time  to  correct  an  error:  13.2  man  min. 


n   of 
Errors 
cor rec 

T 

+ 

29 


25 


ted 


20 


15 


10 


*  • 
** 
*■* 

*  * 

*  • 

*  * 

*  * 


10 


(measured 


-  +  ■ 
15 


■  -  +  ■ 
20 


■  -  +  > 
25 


30 


* 

*  * 

00        120 


TIME  TO  CORRECT  ERRORS 
in  from  ooint  in  time  of  detection) 


180 
man  m  i  nu  t  es 
of  dpo j  ec  t  t  i  me 
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13.  When  errors  were  found: 


a)  *  of  errors  found  during  design  ohase: 

b)  tt    of  errors  found  during  design  review 

c)  *  of  errors  found  during  coding: 

d)  *  of  errors  found  during  debugging: 

e)  tt    of  errors  found  during  writing  of 

test  orocedures: 

f)  *  of  errors  found  during  testing: 


1 

0 

21 

53 

0 
0 

75 


1.3 

0.0 
28.0 
70.7 


0.0  X 
0.0  X 


14.  /then  errors  were  made: 


a)  9  of  errors  made  aurinn  design  phase: 

b)  <  of  errors  made  during  desicn  review; 

c)  *  of  errors  made  during  coding: 

d)  *  of  errors  made  during  debuaging: 

e)  *  of  errors  mane  during  writing  of 

test  orocedures: 

f)  s  of  errors  made  aurina  test ina: 


20 
0 

53 
2 

0 
0 


26.7  X 

0.0  X 

70.7  X 

2.7  r. 

0.0  X 

0.0  X 


75 
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15.  TIME   HISTORY  GRAPHS 


«  of 

Errors 

r 


10* 


5  + 


A 

ccc 

CCAA 


c 

C  A 
A  A 
C  A 

cccc 


C  A  0  0 
OCOO  OCAC  CD  CAODCCO. 
"-■►-•t--  +  -  +  -  +  -+-t-  +  -*-  +  -  +  -  +  -  + 
2u  b0 


Error    Tyoes : 
A:  Clerical  error 
B:  Debugging  error 
C  :  Codi  ng  error 
D  :  Des  i  gn  error 


C 

C 

c 
c 

A 
CA 

BA   D 
CA  CD 


-  +  -  +  -  +  -•»•-  +  -  +  -  +  ■ 
3  0 


112 


125 


man 
hours 


3  of 
Errors 


NUMBER  OF  ERRORS  FOUND  VS  PROJECT  TIME 


10  + 


5  + 


I* 


*  •* 

*  *  « 
»-♦-  + 

24 


*  *  »* 

*  *   **  *  * 

*  *  *  *  *  *  * 


*      ** 
***  **•«*  *  * 
************* 


-  +  -♦-  +  -+-  +  -♦-  +  ->-♦-•♦•-♦-♦-+•-•♦•-  +  -♦-■»•-  +  ->-  + 
60  80 


-  +  -♦• 
112 


•♦->   man 
125   hours 


NUMBER  OF  ERRORS  CORRECTED  VS  PROJECT  TIME 
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ANNEX  F 


ERROR  CATEGORIES  AND  TYPES 


1 .   Des  i  gn  Errors 

The  follwing  types  of  errors  apply   to   both   categories 
"System  Design  Errors"  ana  "Program  Design  Errors": 


01 

02 
03 

Da 

D5 

06 

07 

D8 

09 

010 

011 

Dl2 

D13 

014 

015 

016 

017 


Communication  Error 

Design  Negliaence 

Forgotten  Cases  or  Steps 

Timing  ProO 1  ems 

Errors  in  1/0  Concepts 

Data  Design  Error 

Initialization  Error 

Inadequate  Checking 

Extreme  Conditions  Neglecteo 

Seauencing  Error 

Index  i  ng  Error 

Looo  Control  Errors 

Misuse  of  Boolean  Exoression 

Mathematical  Error 

Reoresen t at i on  Error 

Misunderstanding  of  Problem  Soecifications 

Other  Design  Errors 


Coding  Errors 


CI 

:   M  i  s  u  n  d  e  r 

C2 

Meal  i  g  e  n 

C3 

1/0  Form 

CU 

Mi  so  1  ace 

C5 

Mul t  i  ol e 

C6 

Mi  ss  i  ng 

C7 

Inadeaua 

C8 

Initial i 

C9 

Error  in 

CIO 

Inadeaua 

Cll 

Level  Pr 

C12 

M i ss  i  nq 

C13 

Case  se 1 

C14 

GO  TO  Pr 

C15 

Comment 

C16 

Forgot  t  e 

C17 

Incons  i  s 

CIS 

Wrong  Us 

C19, 

Index  i  ng 

C20' 

Incons  i  s 

C21 

Seauenc  i 

C22. 

Fl ag  Usa 

C23 

Syntax  £ 

C2U 

Looo  Con 

C25 

Incor rec 

C2t> 

Language 

standing  of  Design 

ce 

at  Error 

a  Data  Declaration 

Data  Dec  1 arat  i  ons 
Data  Dec  1 arat  i  on 
te  Data 
zat  i  on  Error 

Parameter  Passing 
te  or  Forgotten  Checking 
ob 1  ems 

Declarations  of  Block  Limits 
ec  t i on  error 
ob 1  ems 
trror 

n  Delimiter 
tency  in  Naming 
e  of  Nested  IF  Statements 

Error 
tent  Use  of 
ng  Error 
ge  Prob I  ems 
rror 

t  rol  Error 
t  Exit  from 


Var i  abl es  or  Data 


SuDrout  i  nes 


Usage  P  '•ob  1  ems 
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ANNEX  F 


ERROR  CATEGORIES  AND  TYPES 


C27:  Forgotten  Statements 

C28:  Representation  Error 

C2*?:  Control  Sequence  Error 

C30:  Incorrect  Subroutine  Usage 

C31:  Other  Coding  Errors 


3.   Clerical 

Al 

A£ 
A3 
A4 


Errors 

Manua 1  Error 
Men t  a  1  E  r ror 

Procedur a  1  Errors 
Other  Clerical  Errors 


Debugging  Errors 


ai 

82 

83 
3a 
85 
B6 


Inappropriate  Use  of  Deougqing  Tools 
Insufficient  or  Inappropriate  Selection 
of  Test  Cases  or  Test  Data 
Misinterpretation  of  Deougginq  Results 
Misinterpretation  of  Error  Source 
Negl i  gence 
Other  Debugging  Errors 


Testina  Errors 


Tl 
T2 
T3 
Ta 
T5 


Inadeauate  Test  Case(s)  or    Test  Data 

Mi  s i n t eroret at i on  of  Test  Results 

Misinterpretation  of  Program  Soec i f i cat i on 

Negl i  gence 

Other  Testing  Errors 
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ANNEX  G 


Pace  1  of  2\ 


PROJECT  a  :   2 
P rogram  oar t  : 


DIRECTED  GRAPH  WE°RESE\'  T  A  T  ION 

CAR1,  CAR2,  CDRl,CDR2,  SE TC AR 1  ,  SETC AR2 , 
SETCDH1,  SETCPR2,  FREE  I ,  F  =  EE2,  SPACE, 
INITIALIZE  ALL,  SET  NAME,  ADD8R0THER, 
BROTHER,  NAM£  OF,  PATH  ID,  NAMF, 
MU'MES  OF  SUCCESSORS,  PREDECESSOR, 
SUCCESSOR,  SET  =  1TH  ID,  PUT  NEXT  PATH- 
HEADER,  \E  <  T  path,  PREVIOUS  PiTH, 
LINK  PiTH  tAC*,  LINK  PATH,  LI'N.  *  FGRaARD, 
LIWKBACK,  DUPLICATE,  ADDNQDE, 
ALTERNATIVE  «AY,  T.M  I  TP  A  Tl-L  1ST 


COMPLEXITY  MEASURES 


NUMBER  OF  STATEMENTS:  3-17 

NUM3ER  CF  NCDES:  2 

NUMBER  CF  ARCS:  I 

(\UfB5R  CF  PATHS:  1 

CYClCMATIC  NUMBER:  V(S)= 


REACHABILITY  CF  NCDES: 
NCC5     1   :     1 
NCCE     2   :     1 


SUM: 


2.CCCCCO 


O 


*■     Error  = 


t-  t  r  ro  r     = 


«3  (in  vijV8ER  OF 
SUCCESSORS) 

U "5  (in  PUT  ^:EjT 
PATHhEADE3) 


0 


NOTE:  All  33  suoroutifes  'i  a  v  e  the  sa^e  structure, 
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ANNEX  G 


9?ce  ?  of 


OIRECTEO  GRAPH  ?EPcESEiMT£TION 
PROJECT  a  :   2 
Proaram  oart  :   ALLOCATE!,  ALL0CATE2 


COMPLEXITY  MEASURES: 


NUMBER    OF    STATE^E'JTS 

NUMBER  CF  NCCES 
NUMBER  CF  ARCS 
NUMBER  CF  PATHS 
CYCLCWATIC  NUMBER 


2 

V  (  G  )  = 


SEAChAeiLITY    CF    NCCES: 


MjC£  1 

NCCE  2 

NCCE  3 

NCCE  A 


SLf:  <..CCCCCC 

REACHABILITY     TNCEX 
CF    CIRSCTEC    Gft^Fh: 


1.CCCCG0 


-    "    (in    Allocate    c) 
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ANNEX  G 


'aae  3  of  2; 


OIPECTED  GRAPH  5£PRESE'\TATI0M 
PROJECT  a  :   2 
Program  cart  :   ERROR 


COMPLEXITY  VEASUPES: 


NUMBER  OF  STATEMENTS:  oa 
MJN9EF  CF  NCC6S:    15 
MJfBcR  CF  ARCS  :    14 
MJ*9  53  CF  PATHS:    13 
CYCLCMATIC  NUM36R:   V(G)= 

TY  CF  NQCSS: 


Sc»lCh« 

eiui 

NCCE 

1 

NGCS 

2 

NCCE 

3 

NCCE 

A 

NCCS 

s 

NCCE 

e 

NCCE 

7 

NCCE 

a 

NCCE 

9 

NCCE 

1C 

NCCE 

n 

NCCE 

12 

NCCE 

i  3 

NOCS 

1a 

NCCE 

15 

SUf.  15.CCCCC 

RE  SChAa  IL  I7Y     INCEX 
CF    OIP.cCTEC    GnAPH  : 


l.COCO-00 


-   iao   - 


ft  N  N  £  X  G 


P  a  a  <»  'j  of  21 


DIRECTED  GRAPH  S  E0C?F  SEN  T  A  T  I  ON 
PROJECT  «  :   2 
Program  cart  :  DIAGNOSE 


COMPLEXITY  MEASURES: 


NUMBER  OP  STATEMENTS:  52 

i\Uf9E3  CP  \  C  C  E  S  :  3 

iSU,"8EP  CF  ARCS  :  9 

.NUfBE^  CF  PATHS:  A 

CYCLCMATIC  MJM8ES :  V(G)= 


REACHABILITY  CF 


,\ac  = 

NCCE 
NC  Cc 
NCCE 
NGCS 
NCCc 
iNCCE 
NCC5 


r<CD£5 

1 

1 

2 

1 

2 

2 

4 


SUf:  17.COOOI 

PEACr-AS  IL  ITY     I:nCcX 
CF    DIReCTEC    GRAPH: 


2.12fOCO 
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a  n  n  e  <  g 


Peoe  5  of  21 


DIRECTED  GRAPH  REPRESENTATION 
PROJECT  a     :  2 
Program    cart  :  SON 


COMPLEXITY  MEASURES: 


DUMBER  OF  STATEMENTS:  15 
NOi»B£R  CF  NCC63:     7 
iS U .* 3  i a  CF  ARCS  :     7 
MJf-ecS  CF  PATHS:     3 
CYCLCfATIC  MJ,yce*:   V(G)  = 

REACHABILITY  Cc  NGDES: 


\GCs 

NQCE 

NCCE 
NCCE 
NCCE 
NCCE 
NCCE 


SL.V:       9.;C3CCC 
REJCHAE  IL  ITY  INCEX 
CF  CIRECTEC  dZAPr  : 


L. 265714 
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fiMvtX  G 


Page  o  n  *    2  1 


DIRECTED  GRAPH  P£PRESE.\T  a  T  ION 
PROJECT  a  :   2 
Program  oarr  :  FIND  ORIGINAL 


COMPLEXITY  MEASURES 


NUMBER  OF  STATEVE! 

NUfBER  CF  NCCbS: 
NUMBER  CF  ARCS  : 
M^BEP  CF  °AThS: 
CVCLCMA7IC  NUMBS* 


TS: 

6 


1  1 


V  <  G  )  = 


REACJ-A6-ILITY 

NGCE  1 

NCCE  2 

rsCCE  3 

NGCE  4 

,NCCc  5 

NOCE  6 


::es 


SUP:  LC.COCOO 

REACHABILITY     INCEX 
CF    CIREC7EC    GRAPH!  i.c6co66 
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ANNEX  G 


Paae    1    of  21 


DIRECTED  GRAPH  REPRESENT  a T I  Oh 
PROJECT  a  :   2 
Program  oart  :  DUPLICATE  CURRENT  PATH 


COMPLEXITY  MEASURES: 


JU^BER    OF    STATEVE>MTS:     15 
iNUfBEP    CF    NCC6S:  5 

NUMBER    CF    ARCS     :  5 

NU>BEF    CF    PATHS:  2 

CYCLC.MATIC     NUMe  =  ?:       V(C)  = 


3E4CFASIL  ITY  CF  NCCE3  : 


NCCE  1 

NCCE  I 

NCCE  2 

NCCE  A 

NCCE  5 


SUf:  7.CC0000 

SEACrAe  IL  ITY     INDEX 
Gc    C  IRECTEC    GRirr  : 


L4CC0GG 


s    an 
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ANNEX  G 


Pace  P  of  21 


DIPECTED  GRAPH  REPRESEN T A T ICM 
PROJECT  a  :  I 
Proaram  oart  :  FIND  PATH 


COMPLEXITY  MEASURES 


DUMBER  OF  STATEMENTS:  12 

NUI^fiER    CF    NCCES:  7 

NUP3E3    CF    ARCS     :  7 

NUfrfiES    CF    PATHS:  2 

CYCLC.^ATIC    NUMBER:       V(G)  = 


SEACI-AeiUTY    CF    NCDE5: 


NCCE 

I 

NCCE 

2 

NGCc 

? 

NCCE 

H 

NCCE 

5 

NCCE 

6 

NCCE 

7 

SUM: 


1C.CCOOO 


3EAOA8  ILITV     INDEX 
CF    OISECTEC    GRAPH: 


L. 428571 
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ANNEX 


3^e  9  o*  i 


DIRECTED  GRAPH  ^PRESENTATION 
PROJECT  u     :  2 
Proaram  part  :  RE'-'OVE  PATh 


COMPLEXITY  '-'easuRES 


NUMHER 

NUNBEP 
Mlf-BES 

,\U*S5R 


OF 

Cr 
Cc 
CF 


STATEMENTS: 

NCCES:  19 
A*CS  :  25 
PAThS:    72 


u5 


CYClCMATIC  NUMBER:   V(G)= 


RE 

NG 

ng 

NO 
NG 
NG 
NC 
\C 
NG 
NG 
NG 
NG 
NG 
NC 
NG 
NG 
NG 
NG 
NC 
NG 


HA8ILITV 

1 
2 


7 

a 
s 

1C 

11 

12 

12 

15 
16 
17 
16 
IS 


CF  NCC5: 

1 

L 

1 
2 

2 
4 


*■  Cr^or  i  47 


iG^: 


37C  .CCCO 


554CFA8 IL ITV  INCEX 
GF  DIPECTtC  C-rAPh: 


19  .47368 
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ANNEX  G 


Paae  10  of  21 


DIRECTED  GRAPH  REPRESENTATION 
PROJECT  u     :  2 
Program  part  :  END  OF  PATH 


COMPLEXITY  MEASURES: 

NUMBER  OF  STATEMENTS:  18 

NUNBER  CF  NCCES:  9 

NUMBER  CF  ARCS  :  11 

NUrBER  CF  PATHS:  9 

CYCLCPATIC  NUMBER:  V(G); 

REACHABILITY  CF  NCCES: 


NQCE 

NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NQCE 
NQCE 


SUP:  25.03000 

SS^CI-ABILITY     INCEX 

CF    DIRECTEC    GRAFr  : 


2.777777 


Error     *     52 
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ANNEX  G 


Paae  11  of    21 


DIRECTED  GPiPM  =?E?JESE'\lTATIPN 
PROJECT  s  :  2 
Proaram  oart  :  INPUT  CI3ECTE0  GRAPH  UFOWMAHON 
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A  N  N  t  X  G 


Pace  1?  of  21 


DIRECTED  GRAPH  REPPESENTAT ICN 

PROJECT  b  :   2 

Proara"-  oart  :  GET  SUCCESSORS 


COMPLEXITY  MEASURES: 

NU*8E°  OF  STATEMENTS:  3a 

MJN8EP  CF  NCCES:  9 

NUV3EF.  CF  ARCS  :  9 

NUNSE9  CF  PATHS:  5 

CYCLCPATIC  NUMBER:  V(G)= 

^SACr-Aeil  ITY  CF  NCC5S: 


NCCE 

1 

1 

NCCE 

2 

1 

rsCCiE 

2 

L 

NCCE 

A 

1 

NCCE 

5 

7 

NCCE 

6 

2 

NGCE 

7 

1 

NCCE 

a 

2 

NGCE 

c 

2 

SUP:       13.CJ0J0 
RE^CrAe  1L  ITY  INCEX 
CF  DIPECTEC  GPJPr: 


1  .444444 


«•  Error  3  3° 
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ANNEX  G 


Page  13  of  21 


QIPECTtQ  GRAPH  REPRESENTATION 

PROJECT  a  :  i 

Progra*  cart  :  OCCURS  TWICE 


COMPLEXITY  MEASURES: 


NUV8EP  OF  STATEMENTS: 
,\C,V6ER  CF  NCCSS:  11 
NU.V3E5  CF  ARCS  :  L4 
M'.V3EP  CF  FATrS:  23 
CYCLC»ATIC  NUMBER  :   V  ( 


11 


S6A( 
iNuC! 
NCCi 

nccs 

NCC! 
NCCi 
\CC; 
NCCi 
NCC! 
NCCi 
NCCi 
NQCi 


•  3ILITY 
1 
2 


6 
7 
8 
9 
1C 
11 


CF 


NCCi 
I 
1 
7 


6 

7 

7 

14 

14 

28 


CMU  i 

PSAOAeiL  1TY 

3F  DIRSCTEC  GRAPH 


93.CC0C0 
INCcX 


3  .45454! 
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ANNEX  r, 


Peoe  l'-i  of     2 


DIRECTED  GRAPH  REPRESENT AT  ION 
PROJECT  *  :  2 
Program  nart  :  0  iv4  E  <n  A  Y 


COMPLEXITY  v£iSu«ES: 


NUMBER    OF    STATEWE"JT3:     1  1 
<\U,"eER    CF    \CCES:  4 

MJPBE*    CF    ARCS     :  <+ 

i\Ur3ER    CF    PATHS:  I 

CYCLC,VATIC    NUMBER:       V(GJ 


REACHABILITY    CF    ISCDES 


NQCE 

1 

1 

NCCE 

2 

!               2 

\C  Cc 

■2 

1 

NCCE 

4 

2 

SUf:  6.CC00D0 

REACHABILITY     INCEX 
CF    CtRECTEC    GRAPH  : 


l.sccoca 
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iiNJNEX     G 


Pace     15    o'     21 


OI^cCTEO  GRAPH  atP«F3t^T  JTIG'^ 
PROJECT  s  :  ? 
Proqran-  oart  :  VATCHING 
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ANNEX  G 


P a q e  lb  o  f  2  1 


DIRECTED  GR4PM  REPRESENTATION 
PROJECT  a  :   2 
Program  oart  :  CHEC«RACX 


a  )  3  o  f    nodes :  1 5 

ft)  2  of  arcs  :  la 

c)  3  of  statements:  3  n 

-;)  3  of  oafs:  * 

e)  Reachability:  * 

f)  CyClOmaMC  nij»nar;   5 
*  H u  m D  e  r  of  oitis 

and  reacnao'lity 
are    very  I  ?n». 
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AMNEX     G 


Pace     17    0f    2] 


DIRECTED    GRAPH    -?EDPESEfjTATIQM 
PROJECT    =     :       2 
Program    oarr.     :    S«VE    MEMORY 


COMPLEXITY    V'EASUPES: 

MUMPER    OF    STATEME'IT3:     ^u 
NUMBER    CF    NCC  ES  :  6 

NU^&cR     CF    ARCS     :  6 

i\Uf5ES    GF    PATHS:  2 

CYCLCM4TIC    NUMBER:       V(G)  = 


RE^CrABILI 

TY  CF 

NQCcS: 

mdc?    i 

1 

nccs    2 

I 

NCCS     3 

1 

NCCc      4 

1 

NCCE     5 

2 

NGCi     fc 

2 

SUM: 

3 

.CCCCOC 

REACrAE  IL  I 

T 

r  INCEX 

CF  0IREC7E 

c 

33 
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ANNE*  G 


Page  1«  of  21 


DIRECTED  GS4PH  REPRESENTAT  ION 
PROJECT  a  :   2 
Program  nam  :  LIST  PATH 


COMPLEXI TY 

MEASURES: 

NUMBER 

OF 

S 

r  a  te 

**ENT3:    2"> 

NUfBEP 

CF 

NCCES 

:          17 

iSUi^EcS 

CF 

.4SCS 

:          19 

NUN8ER 

CF 

FATFS 

:         12 

CVCLCMATIC 

MJr*C 

E  S  :       V  (  G  J  =           ■+ 

SE^CrABIL I 

TV    CF 

NCCES : 

NCCE 

1 

I 

NCCE 

2 

1 

iNCCE 

"2 

1 

NGDE 

A 

1 

NGCE 

c 

2 

NCCE 

6 

2 

NCCE 

7 

2 

NC  CE 

3 

2 

NCCE 

9 

2 

NCCE 

IC 

2 

NCCE 

11 

A 

NCCE 

12 

2 

NCCE 

12 

5 

NGCE 

14 

4 

NCCE 

15 

A 

NGCE 

16 

E 

NGCE 

17 

: 

£ 

SUf1 

5 

:.  coccc 

^E^CHAE  IL  IT> 

'     INC  E  X 

GF    CIRE 

CTE 

r 

GSAPr:                    2.94U76 
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ANNEX  G 


Paa< 


.9  of  21 


DIRECTED  '>Si?u  REPRESENTATION 
PROJECT  a  :   2 

Program  oart  :   LIST  ALL  P  A  T  r>  S  ,  MAIN 


COMPLEXITY  MEASURES 


NUMBER  OP  STATEMENTS 
NUMBER  Cf  NGCE5:  *> 
NUMBER  CF  ASCS  :  4 
iNU>6ES  CF  PAThS:  2 
CYCLCMaTIC  NUMBER:   VlG) 


REACHABILITY  CF  NCOES: 


10-11 


NGCE 
NCCc 
NGCS 
NQCs 


2 


SUV:  6.CCOC0O 

REACHABILITY     IMCEX 
CF    OIRSCTEC    «3APh  : 


l.SOCCCO 
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ANNEX  G 


Page  20  of  2 


DIRECTED  GPAPh  3EP<*ESE\T  ATIQim 
PROJECT  s     :   2 
Program  oart  :  ANALYZE 


COMPLEXITY  ^EASUWES: 


NUMB£P 

OF  STATE'-'E^rS:  2  3 

NUN6  ER 

CF  NCCES:     7 

MJNBER 

CF  ARCS  :     3 

NUMBER 

CF  PATHS:     •♦ 

CYCLCMATIC  MJf6E?  :   V(G)  = 

REACrA; 

ilLlTY  CF  NCDSS: 

NGCE 

1 

1 

NGCE 

2 

1 

\oce 

- 

1 

NGCE 

4 

2 

NGCE 

5 

4 

NGCE 

6 

2 

NGCE 

7 

i. 

SUN:  15.COC0O 

REACFAEILITY     INCEX 
OF    OIRECTEC    GRAPH:  2.142257 
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ANNEX  G 


Pace  2  I  or  2  1 


DIRECTED  GRAPH  REP^ E SE MTA T I 0M 
PROJECT  a  :  2 
Program  oart  :   SET  PATHS 

a)  ■*  of  noaes:  JO 

p )  a  of  arcs  :  5a 

c)  •    of  statements:  S  a 

d)  =  of  oaths :  * 

e )  Reachaoi  H  t v  J  * 

f)  Cyc I omst  i  c  number:  1 
*  M umber  of  oafis 

and  reachability 
are  very  la  roe . 
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ANNEX    H 


Page    1      of    2 


TEST    PHASE    DESCRIPTION 


Project    #    :    2 

Test  run  «  :  1         Including  I   Test  Step 

Begin  of  Test  (day/time)  :  03/17/1500     End  of  Test  (day/time)  :  03/17/1900 


CPU  time  for  necessary   comoiles  (in  sec.):  7.53 
a)   7.53   o)        c)        a)       e)         f) 

CPU  time  for  TEST  run  (sec)  :  25.29 


g) 


Man  Hours  for  this  Test  run  :  3.0  (including  oreoaration   of 
tests) 


TEST 
STEP 


OBJECTvE 


EXPECTED 

RESULT 

(TOLERANCE) 


ACTUAL 
RESULT 


1) 

ERROR 


DAY 

TIVE 


COMMENTS 

AND 
CODED  ERROR 
TYPES 


Test  performance 
of  program  for  va- 
r  i  ous  di  rec  tea 
graphs : 

a ) di  rec  t  ed  graph 
9  nodes,  10  arcs 
b  )di  rect ed  graon 
6  nodes,  7  arcs 
c )di  rec ted  graph 
b    nodes,  7  arcs 
d)di  rect  ed  graph 
22    nodes,  25  arcs 
e )di rec  t  ed  graph 
22    noaes,  27  arcs 
f )di  rec ted  graph 

1  noae,  1  arc 
(self  1 ooo  at  node 
1) 

g) di  rect  ed  graph 

2  noaes,  2  arcs 

h ) di  rect  ed  graph 
2  nodes,  1  arc 


valid  paths 
valid  paths 
va 1 i  a  oaths 
valid  oaths 
valid  paths 


no  valid  oath 
wa rn  i  ng  shou 1 d 
be  Drintea 

2  valid  oaths 

1  valid  path 


O.K. 

O.K. 
O.K. 
O.K. 
O.K. 
O.K. 

O.K. 
O.K. 


3/17 

1500 


1) 


Recora  -nen 
error  occurs. 


Remarks:  Directed  graphs  submitted  for  testinq 
boundary  conditions  and  special  cases 
to  be  difficult  to  examine. 


3/17 

1900 

i  nc 1 ude  a  variety  of 
which  are  considered 
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ANNEX  H 


Page  2   of  2 


TEST  PHASE  DESCRIPTION 


Project  *  :  2 

Test  run  U    :  1         Including  1   Test  Steos 

Begin  of  Test  (day/time)  :  03/15/1200     End  of  Test  (day/time)  :  03/17/2200 


CPU  time  for  necessary   compiles  (in  sec.):  fa. 83 
a)   6.83   b)        c)        d)       e)         f) 

CPU  time  for  TEST  run  (sec)  :  8UU.UU 


g) 


Man  Hours  for  this  Test  run  :  10.0  (including  preparation  of 
tests ) 

1  ) 


TEST 

OBJECTVE 

EXPECTED 

ACTUAL 

ERROR !DAY 

COMMENTS 

STEP 

RESULT 
(TOLERANCE) 

RESULT 

a   ITir-'E 

AND 

CODED  ERROR 
TYPES 

Find  limits  of  the 
orogr am  using  a 
directed  graon  ha- 
v  i  ng  an  infinite 
number  of  oaths 
according  to  the 
definition  bei  ng 
used . 


Program  should 
list  all  oaths 
un t  i 1  f  r ee 1 i  s  t  ^2 
is  exhausted. 
Uoon  termination 
of  the  Drogram  an 
appropriate  warn- 
i  ng  shou 1 d  oe 
pr i  nted#  such 
that  the  user  can 
interpret  the  re- 
sults of  the  oath 
analysis. 


O.K 


3/15 
1200 


3/17 
2200 


1)  Record  wnen 
error  occurs, 


Remarks:  Due  to  extensive  debugging  previous  to  testing  no  errors 
have  been  found  during  the  testing  phase. 

Test  data  for  this  run  are  chosen  to  indicate  limitations 
of  the  program  oecause  of  the  finiteness  of  time  and  memory 
available. 


-  160  - 


APPENDIX  C 
PROJECT  DESCRIPTION 


Project  n    :  3 


Project  title:   PATH  ANALYSIS  IN  DIRECTED  GRAPHS 

WITH  RESPECT  TO  REACHABILITY  OF  NODES 

Programmer  :  HOFFMANN 

Programming  Language  :  ALGOL 

Programming  environment:  IBM/360/67, OS/MVT , BATCH 

Design  notes  :  see  ANNEX  A 

Program  listing  :  see  ANNEX  B 

Coding  notes  :   see  ANNEX  C 

Debugging  notes  :   see  ANNEX  D 

Error  Listing  :    see  ANNEX  E 

Final  statistics  :   see  ANNEX  F 

Graphical  representation  :  see  ANNEX  G 

Test  phase  description:  see  ANNEX  H 


Starting  date:  1  MAR  77 


Ending  date:  17  MAR  77 


EXPERIMENT  DESCRIPTION 

1.  Project  description: 

This  program  is  an  extension  of  project  tt  2 .   The   graph 

analyzing   program   as  developed  in  project  tt  2    was  modified 

to  calculate  the  reachability  of  each  node.  A   summary   of 

the  calculations  is  printed  at  the  end  of  the  analysis.   All 
features  described  in  project  #    2    will  be  unchanged. 
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Input:  via  punch  cards 

Output!  via  line  printer 

(For   input   description   and   error   nnessages   see   project 

descriotion  of  project  U    2.) 
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2.  Programmer's  background: 

a)  Experience  in  programming: 

Oct  1970  -  May   1971    Programming  courses 

May  1971  -  April  1972   Module  Programmer 

May  1972  -  June   1974   Work  in  Test  and  Simulation  Depart- 
ment at  the 

NAVAL  COMMAND  AND  CONTROL  SYSTEMS 
COMMAND  (FEDERAL  GERMAN  NAVY) 
Testing  of  tactical  real  time  systems 

March  1975  -  Jan  1977   Student  at  the  NAVAL  POSTGRADUATE 

SCHOOL,  Monterey,  Computer  Science 

b)  Experience  in  testing: 

Two  years  of  work  in  testing  and  simulation. 

c)  Experience  in  the  area  of  the  given  problem:   None. 

d)  Experience  in  the  progammina  language  being  used: 
Experience   over   a   period  of  IB  months  in  more  than  10 

programming  projects.   (Total  number   of   source   statements 
produced  during  that  time  was  more  4000.) 

3.  Psychological  factors: 

a)  Did  the  programmer  like  the  project?   Yes. 

b)  How  does  the  programmer  like  the  programming 
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1 anguage? 
Favorite  programming  language. 

c )  Was  the  programmer  satisfied  by  the  way  the  problem 
was  spec  i  f  i  ed? 

Only  minor  criticism. 

d)  How  did  the  programmer  like  the  programming  environ- 
ment ? 

The  facilities  (study  room,  card  punch  room)  were  not 
conducive  to  efficient  orogramfning  because  of  restricted 
space/  bad  lighting  and  noise. 

e)  Other  factors: 

The  recording  of  the  experiment's  data  during  the  pro- 
ject affected  speed  and  concentration  considerably. 
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U.    Comments  on  Documentation  s 

For  the  documentation  of  each  software  development  chase  a 
special  documentation  form  has  been  developed.  These  forms 
are  designed  to  provide  a  firm  guideline  for  the  experiment 
programmer  to  record  all  data  of  interest  for  subsequent 
error  ana  lysis. 

-  Begin  and  end  of  each  step  was  recorded  with  resoect  to 
day  and  time. 

-  Each  error  was  recorded  when  it  is  discovered.  The  error 
was  then  identified  by  a  unique  error  number  (1/2/...). 
Furthermore  the  time  of  discovery  and  the  error  type  (using 
error  types  listed  in  ANNEX  F)  were  recorded. 

-  If  appropriate/  comments  about  error  discovery/  reason  why 
the  error  was  made/  etc.  were  documented  in  ANNEX  E2. 

-  For  each  error  the  phase  in  which  the  error  was  made/  the 
phase  in  which  the  error  was  discovered  and  the  time  spent 
to  correct  the  error  was  recorded  in  ANNEX  El. 

For  each  step  in  any  one  of  the  software  development 
phases  the  day/time  of  begin  and  end  was  recorded.  In  addi- 
tion/ the  time  (in  man  hours)  for  each  step  was  recorded. 
This  excludes  the  overhead  used  for  documentation  of  the 
exper  i  ment  data. 
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ANNEX  A 


Page  1   of  1 


WORKSHEET  FOR  DESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  n 


STEP 

ft 


PROBLEM  AND  PLANNED  SOLUTION 


ALTERNATE 
SOLUTIONS 


!  MAN  : 

DAY  JHOURS'.ERROR 
TIME! /STEP!   U 


COMMENT 


Analyze  ways  to  comoute  the 
reachability  inoex  for  each 
node . 

-In  general  the  reachability 
index  of  a  node  is  incremented 
by  1  when  the  node  is  added  to 
the  current  oath  or  out  as  the 
first  node  on  an    alternative 
oath. 

-The  reachaoilitv  of  nodes 
which  belong  to  a  oath  which 
has  to  be  removed  has  to  oe 
cecremented  by  1  uo  to  tie 
node  whic"  has  been  the  last 
branch  ooint  on  this  oath  ex- 
cludina  the  branch  ooint 
i  tsel  f . 


Too-Oown  Design: 

a)  Define  additional  subrou- 
t  i  nes : 

-  Uodate  Reachability 
( i  nc  rement  by  I  ) 

-  Correct  Reachability 
(decrement  by  1  for  all  ore- 
vious  nodes  uo  to  and  exclu- 
dina  the  last  branch  ooint) 

-  List  Reachabi 1  i  t  y 
(output  of  resul t  s  ) 

b)  Define  necessary  chances 
within  current  version 

of  the  analyzing  orogram. 


3/19 

1500 


3/19 
1800 

3/20 

160  0 


3/20 
2000 


3.0 


4.0 
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Annex  B 


Program  Listing  of  Projects  U    2    and  3 
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ANNEX  C 


Page  1  of  1 


WORKSHEET  FOR  COOING  PHASE  OF  PROJECT  ft 
Beginning  of  Coding  (day/time)  :  3/20/2000 
End  of  Coding    (day/time)  :  3/21/1200 
Man  hours  :  <i .  0  (including  ounching  of  cards) 


COOING 
BEGIN      !     ENO 
DAY/TIME   j  DAY/TIME 


PROGRAM  PART 


1) 

ER"0R!DAY 
ft   iTIME 


COMMENT 


03/20/2000 


03/21/1000 


03/20/2200 
03/21/1200 


■Changes 
■Subrou t i  nes 
(new ) 

»°unc  h i  na  cards 


1050 


1)  'ecoro  when  error  is 

detected. 


Cl<? 
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ANNEX  0 


Page  1  of  1 


rtORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  n    :  3  DE8UG  Run  *  :  1 

Begin  of  Debug  Run  (day/time)  :   03/21/1200 

End  of  Deoug  Run    (day/time)  :   03/21/1520 

*  of  Debug  Steps  incl.  in   Debug  Run:  5    CPU  time  for  Debug  run  (sec):  41.13 

CPU  time  for  necessary   comoiles  (sec)  :  29.98 

a)   6.03    b)   8.«3    c)   7.65    a)  7.87    e)  f)  g) 

Man  hours  for  this  Debua  Run  :  3,0  (including  preparation  of  debua  run) 

man        1  ) 


STEP 

3 


PROGRAM 
PART 


OBJECTIVE  AND    EXPECTED 
"ESULT 


ACTUAL 
RESULT 


DAY  irtOURS [ERROR 
TIME!/STEP!   * 


COMMENTS 
AND  CODED 
ERROR  TYPES 


New 

subrou- 

t  i  nes 

and 

program 

changes 


•Get  error  free  comoile 


•repeat  step  1 


■repeat  step  1 


-Check  reachability  cal- 
culation for  trivial  cases 


-repeat  step  4 


1  comoi 1 e 
error 


1  pro- 
gram 
error 


O.K. 


wrong 
branch- 
poi  nt 
ca  1  cu  1  a* 
t  i  on 

O.K. 

/ 


3/21 

1215 


3/21 
1315 


3/21 

1345 

3/21 

1400 


3/21 
1450 

3/21 

1520 


1  .0 


0.5 


0.3 


0.7 


0.5 


1  )  Record  when 

error 

occurs 
C6 

( forgot  t en 
dec  1 arat  i  on 
of  variable) 


C24 


C21 
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ANNEX  E 


Page  1   of  1 


ERROR  LISTING 
PROJECT  *  :  3 

Begin  of  Project  (day/time)  :  03/19/1500 
End  of  Project  (day/time)  :  03/23/2300 
Man  hours  for  total  project  :  33.0 


ERROR 
tt 

PHASE 

in  which 

ERROR  was 

di  s- 

covered 

PHASE 
in  which 
ERROR  was 
made 

ERROR 

TYPE 

(see  ANNEX  F) 

TIME 
scent  to 
solve  the 
ERROR 
(Man 
Jiin.) 

*    of  OTHER 
STATEMENTS  OR 
PARTS  OF  THE 
PROGRAM 

AFFECTED 

1 
2 
3 

a 

Cod  i  ng 
Debuggi  ng 
OeDuggi  nq 
Deouggi  ng 

Codi  ng 
Codi  ng 
Codi  ng 
Coding 

Cl° 

C6 

C2a 
C21 

10 

5 

5 

20 

ERROR  LISTING  (COMMENTS) 


ERROR 

n 

DAY 
TIME 

COMMENTS 

(EVIDENCE,  THOUGHTS,  WHY  WAS  THE  ERROR 

WHY  AND  HOW  WAS  THE  ERROR  DISCOVERED? 

ERROR  BLOCKING,  etc.) 

MADE? 

1 

2 
3 

u 

03/21 

1115 
1215 
1315 

1U00 

Checking  code  while  ounching  cards. 
Found  through  comoiler  diaanostics. 

Found  bv  interoreting  results  of  debug 

run . 

-   19a   - 


ANNEX  F 


FINAL  STATISTICS 


PROJECT  *   3 

FINAL  STATISTICS 

Project  name  :  PATH  ANALYSIS  IN  DIRECTED  GRAPHS 

rtlTH  RESPECT  TO  REACHABILITY  OF  NODES 

Short  description: 

This  project  is  an  extension  of  project  U  2.  The  graph 
analyzing  program  developed  in  project  *  2  i  s  modified  to 
calculate  the  reachability  of  each  inaivioual  node.  At  the 
end  of  the  analysis  the  reacnaoility  inaex  of  each  node  is 
printed.  All  features  of  tte  program  being  changed  are 
preserved. 
Input:  via  punch  cards  Outout:  via  line  printer 

Quantitative  measures: 

1.  9    of  source  statements  :  70  (including  necessary  changes) 

2.  Total  man  hours  for  project  :  33.0 

3.  Man  hours  soent  in 

a)  Des  i  gn  :  7.0 

b)  Coding  :  4.0 

c)  Debugging  :  3.0 
a)  Testing  :  19.0 

a.  CPU  time  for  compiles:  5a. 08  sec. 

5.  CPU  time  for  debug  runs:  11.13  sec. 

6.  CPU  time  for  test  runs:  391. U8  sec. 

7.  it    of  test  and  debug  runs:  4 

8.  *  of  test  and  debug  steps:  8 

9.  =  of  errors  found:  4 

10.  Total  man  hours  used  to  correct  errors:  0.7 


-  195  - 


ANNEX  F 


FINAL  STATISTICS 


It.  Error  Dec ec  t  i  on: 

Mean  time  between  error  detections:  43.3  man  min. 
U    of 
errors 
t 


5  ♦ 


1  ♦ 


**«*****#««******** 


---.--_+.----_  +  _-_--  +  _-___  +  _-_--  +  _-.••  +  -----»-----*-----  +  -- 
15    3  0    4  5    6  0    7  5    R  0    105    120    135 
TI«E  BETWEEN  ERROR  DETECTIONS 
(measured  from  ooint  in  time  of  detection  of  orevious  error) 

12.  Error  Correction: 

Mean  time  to  correct  an    error:  10.0  man  min. 

a  of 
Errors 
cor rec  t  ed 

r 


■>  man  min  of 

pro  j  ec  t  time 


5  + 


1  ! 
-  +  « 

1 


10    15    20    25    30    35 


■-  +  • 

40 


TIME  TO  CORRECT  ERRORS 
(measured  in  from  ooint  in  time  of  detection) 


man  minutes 
of  pro j  ect  t  i  me 
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ANNEX  F 


FINAL  STATISTICS 


13.  When  errors  were  found: 


a) 

b) 
c) 
d) 
e) 


f)  « 


of  errors  found  during  design  Dhase; 

of  errors  found  during  design  revie* 

of  errors  found  during  cooing: 

of  errors  found  during  debugging: 

of  errors  found  during  writing  of 

test  procedures: 

of  errors  found  during  testing: 


0.0 

0.0 

25.0 

75.0 

0.0 
0.0 


14.  When  errors  were  made: 


a)  3  of  errors  made  during  design  chase: 

b)  s  of  errors  made  during  design  review: 

c)  *  of  errors  made  during  coding: 

d)  *  of  errors  made  during  debugging: 

e)  «  of  errors  made  during  writing  of 

test  orocedures : 

f)  «  of  errors  made  during  testing: 


0 

= 

0.0 

X 

0 

s 

0.0 

% 

a 

= 

100 

X 

0 

= 

0.0 

z 

0 

3 

0.0 

•; 

0 

= 

o.o 

•: 
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ANNEX  F 


FINAL  STATISTICS 


15.  TIME   HISTORY  GPAPHS 


#  of 
Errors 

T 


5  + 


Error  Types: 
A:  Clerical  error 
B:  Debugging  error 
C :  Coding  error 
0:  Design  error 


c  c  c  c 

•t"+"+-+"+"t»+«t«+-+»+"+»+"+«+"+«t«+»+"+»+«+«+"+«+"+»+«t"+»+-+»+»+">  man 

12    3    4    5                      10                      15                      20                      25                      30            33         noun 
. . .DESIGN CODING. .DEBUGGING.. TESTING 

(soft-are  aevelooient  ceases) 


NUMBER  OF  ERRORS  FOUND  VS  PROJECT  TIME 


«  of 
Errors 

T 

I 
I 

5  + 


1*  C   C  C   C 

-♦-  +  -♦-♦-♦-♦-  +  -*-  +  -♦-  +  -♦-♦-♦-♦-♦-  +  •♦-♦«  +  -♦-  +  -♦-*-+•-  +  -  +  -  +  -♦-  +  -♦•-  +  -  +  -•>  man 
12  3  4  5         10         15         20         25         30     33    hour; 

NUMBER  OF  ERRORS  CORRECTED  VS  PROJECT  TIME 


-  198  - 


ANNEX  F 


ERROR  CATEGORIES  AND  TYPES 


Design  Errors 

The  follwing  tyoes  of  errors  aooly   to   both   categories 
"System  Design  Errors"  and  "Program  Design  Errors": 


Dl 
02 
03 

Da 

D5 

06 

07 

08 

09 

010 

011 

D12 

013 

014 

015 
016 
017 


Communication  Error 

Oesign  Negligence 

Forgotten  Cases  or  Steps 

T  i  m  i ng  P  rob  I  ems 

Errors  in  1/0  Concents 

Oat  a  Des  i  gn  Error 

Initialization  Error 

Inaaequate  Checking 

Extreme  Conditions  Neglected 

Sequencing  Error 

Index  i  ng  Error 

Looo  Control  Errors 

Misuse  of  Boolean  Exoression 

Mathematical  Error 

Representation  Error 

Misunderstanding  of  Problem  Scec i f i cat i ons 

Other  Design  Errors 


Codi  ng  Errors 


CI 

C2 

C3 

C4 

C5 

C6 

C7 

C8 

C9 

C10 

Cll 

C12 

C13 

C14 

CIS 

C16 

C17 

C18 

CIO 

C20 

C21 

C22 

C23 

C24 

C25 

C26 


Mi  su 

Neal 

1/0 

Mi  sp 

Mul  t 

Mi  ss 

Inad 

Ini  t 

Er  ro 

Inad 

Leve 

Miss 

Case 

GO  T 

Comm 

Forg 

Inco 

Wron 

Inde 

Inco 

Sequ 

Flag 

Synt 

Loop 

Inco 

Lang 


nder 
i  gen 
Form 
1  ace 
i  c  1  e 
i  ng 
egua 
{all 

r  i  n 
eaua 
1  Pr 
i  ng 

sel 
0  Pr 
ent 
ot  te 
nsi  s 
g  Us 
x  i  ng 
nsi  S 
enc  i 

Usa 
ax  E 

Con 
r  r  ec 
uaae 


stan 
ce 

at  E 
d  Da 

Oat 
Oata 
te  D 
zat  i 

Par 
te  o 
oble 
Oecl 
ect  i 
obi  e 
Er  ro 
n  De 
t  enc 
e  of 

Err 
tent 
ng  E 
ge  P 
r  ror 
t  rol 
t  Ex 

Usa 


ding  of  Des  i  gn 


or 

Dec  1 ara  t  i  on 
Oec 1 arat  i  ons 
ec 1 arat  i  on 
a 

Error 
eter  Passing 
Forgotten  Checking 

at  tons  of  Block  Limits 
error 


miter 

in  Naming 

ested  IF  Statements 

se  of  Variables  or  Data 

or 

b  1  ems 


Error 
it  from  Subroutines 
qe  Prob 1  ems 
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ANNEX  F 


ERROR  CATEGORIES  AND  TYPES 


C27:  Forgotten  Statements 

C28:  Representation  Error 

C29:  Control  Sequence  Error 

C30:  Incorrect  Subroutine  Usage 

C31:  Other  Coding  Errors 


Clerical  Errors 


Al 

A2 
A3 

Ay 


Manual  Error 
Ment  a  1  Error 
Procedural  Errors 
Otfier  Clerical  Errors 


Debugging  Errors 

81  :  Inaooroor i at e  Use  of  Debugging  Tools 

&d  :  Insufficient  or  InaooroDr i a t e  Selection 

of  Test  Cases  or  Test  Data 

63  :  Misinterpretation  of  Debugging  Results 

8^  :  M i s i n t ero r e t a t i on  of  Error  Source 

Bb  :  Negligence 

86  :  Otier  Debugging  Errors 

Test  i  ng  Errors 


Tl 
T2 

T3 
Tu 
T5 


Inadequate  Test  Case(s)  or  Test  Data 

Misinterpretation  of  Test  Results 

^i s i n t e ro re t at i on  of  Program  Specification 

N  e  g  1  i  g  e  n  c  e 

Other  Testina  Errors 
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ANNEX  G 


Page  1  o  f     3 


DIRECTED  GRAPH  REPRESEf'i  T  A  T  TON 
PROJECT  a  :   3 
Program  oart  :  CORRECT  RE  ACh  AB  1 1_  I  T  i 


COMPLEXITY  MEASURES 


NUMBER  OF  STATEMENTS:  10 

NUVP  ES    CF    NCC £S :  4 

\U*55S    CF    A?CS     :  <* 

MJN5SR    CF    PATHS:  2 

CYCLCKATIC    NU^SSR:  v(G)= 


5EACHA2ILITY  Cr    NC3E: 
NOCS            1       :  1 

i\CCE  2       :  2 

HCCz  3       :  1 

ISQCc  k       •  2 


SUM: 


6 • C  CO 033 


REACHABILITY     INCsX 
CF    OISECTEC    GPAPI-: 


l.SCCOOO 
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ANNEX  G 


Paae  2    o*     3 


DIRECTED  GRAPH  REPRESENTATION 
PROJECT  «  :   3 
Program  cart  :  UPOAfE  RE AC^AH IL I TY 


COMPLExITf  MEASURES: 


NUMBER  UF  STATEMENTS:  3 

.NUfEES    CF    NCCcS:  2 

NU>5£S    CF    A,«CS     :  L 

,\U>?cft    CF    PATHS:  1 

CYCLCiVATIC    NU1*3EP:  V(G)  = 


PSAd-AEILITY    CF    f^COES: 
NCCE  1       :  1 

NCCE  2       :  1 

SUM:  2.CCOOOO 

3£>iOAa  IL  I  TY     INCEX 

CF    D  IRECTEC    GF-AFr  : 


L.CCJOOO 


o 


© 
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ANNEX  G 


?dae  3  of  3 


DIRECTED  GRAPM  REpPE5£.'-J  T  A  T  I  CN 
PROJECT  a     :        3 
Program  rart  :   LIST  REACHABILITY 


COMPLEXITY  MEASURES: 

•MUveER    OF    STA  TE^EM  TS  :     1  3 
MJPBER    CF    i\CC£S; 
NU^BES    CF    iftCS 
MJfBEP    CF    PATHS 
CYCLCVATIC    MJPeES:       V(C)  = 

SEAChAElLITY  CF  ,NCDc3: 


.NOCE 

1       : 

1 

NCCE 

2       : 

2 

NCCE 

2            ; 

1 

bZCZ 

<*       : 

2 

„„      SUM:  c.CC^CCC 

«t£ChAP  IL  ITY     INDEX 
CF    DIRECTED    GPAPr  : 


1.5GOGJ 


«■     Error     a     3 
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ANNEX  H 


Page  1   of  3 


TEST  PHASE  DESCRIPTION 


Project  #    :  3 

Test  run  »    :  1         Including  1   Test  Step 

Begin  of  Test  (day/time)  :  03/22/1000     End  of  Test  (day/time) 


03/22/1800 


CPU  time  for  necessary   comoiles  (in  sec):  8.33 
a)   8.33   b)        c)        d)       e)         fj 

CPU  time  for  TEST  run  (sec)  :  80. 74 


g) 


Man  hours  for  this  Test  run  :  8.0  (including  oreoaration   of 
tests) 

1  ) 


TEST!      C8JECTVE 

EXPECTED 

ACTUAL {ERROR 

DAY 

COMMENTS 

step; 

RESULT 
(TOLERANCE) 

result;  n 

TI^E 

AND 
CODED  ERROR 
TYPES 

1  )  Record  when 

error  occurs . 

1   ! Comoare  resu 1 t  s 

Program  oerfor- 

O.K.  ! 

3/22 

!  of  prev  i  ous  test 

mance  should  not 

1000 

!  runs  with  results 

be  changed  with 

!of  modified  vers- 

resoect  to  oath 

lion  of  the  orogram 

analysis  as  being 

! us i  ng  same  i  nout 

soec  i  f  i  ed  in  oro- 

{data. 

ject  «  2. 
Addi  t  i  ona 1  ou t - 
out  (such  as  cy- 
c  1  omat  i  C  number 
ana  reachabi lity 
i  ndex )  must  be 
ver  i  f  i  ed  by  desk 
Checking. 

3/22 

1800 
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ANNEX  H 


Page  2   of  3 


TEST  PHASE  DESCRIPTION 

Project  *  :  3 

Test  run  u    :  2         Including  1   Test  Stec 

Begin  of  Test  (day/time)  :  03/22/1800     End  of  Test  (day/time)  :  03/23/1800 


CPU  time  for  necessary   compiles  (in  sec):  7.90 
a)   7,90   b)        c)        d)       e)         f) 


g) 


CPU  time  for  TEST  run  (sec) 


139.00 


Man       Hours  for  this  Test  run  :  8.0  (including  preoaration  of 
tests) 


1) 

TEST!      OBJECTVE 

expected 

ACTUAL 

ERR0R1DAY 

COMMENTS 

step: 

RESULT 
(TOLERANCE) 

RESULT 

4        J  TIME 

AND 
CODED  ERROR 
TYPES 

1  )  Reco rd  when 

error  occurs. 

1   ICompare  hand  cal- 

In cases  of  di  s- 

O.K. 

13/22 

culated  results  of 

creoancies  pro- 

! 1800 

JNTDS  subroutines 

gram  should  oro- 

!  o  f  modu  1  e  1  «*  i  t  h 

v i de  the  correct 

!  program  resu 1 t  s . 

answer  otherwise 

!  Check  for  disc  re- 

resu 1 t  s  must 

! pane  i  es  (if  any ) . 

match.  In  case  of 
discrepancies  re- 
sults must  be 
ver i  f  i  ed  by  desk 
check  i  ng. 

13/23 
!  1800 
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ANNEX  H 


Page  3   of  3 


TEST  PHASE  DESCRIPTION 

Project  *    :  3 

Test  run  U    :  3         Including  1   Test  Step 

Begin  of  Test  (day/time)  :  03/23/2000     End  of  Test  (day/time)  :  03/23/2300 


CPU  time  for  necessary   compiles  (in  sec.):  7.87 
a)   7.87   b)        c)        d)       e)         f) 

CPU  time  for  TEST  run  (sec)  :  171.7a 


g) 


Man  Hours  for  this  Test  run  :  3.3  (including  oreoaration   of 
tests) 


TEST 
STEP 

08JECTVE 

EXPECTED         [ACTUAL 
RESULT           [RESULT 
(TOLERANCE)      [ 

1) 

ERROR 
a 

DAY 
TIME 

COMMENTS 
AND 

CODED  ERROR 
TYPES 

1 

Compare  hand  cal- 
culated resu Its  of 
NTDS  subrout  i nes 
of  modu 1 e  2  with 
Drogram  resul t  s  . 
Check  for  disc  re- 
panc  i  es  (if  any  )  . 

In  cases  of  dis-  !  O.K. 
c  reoanc  i es  oro-   ! 
gram  shoula  oro-  ! 
y  i  de  the  cor rec  t  ! 
answer  otherwise  ! 
resul t S  must       ! 
match  .  In  case  of  ! 
ai sc reoanc i es  re-! 
su 1 t  s  must  be     ! 
verified  by  desk  [ 
Checking.          [ 

3/23 

2000 

3/23 

2300 

1  )  Record  when 
error  occurs . 
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APPENDIX  D 

PROJECT  DESCRIPTION 

Project  n    :  4 

Project  title:   DATA  RETRIEVAL  SYSTEM 

Programmer  :  HOFFMANN 

Programming  Language  :  ALGOL 

Programming  environment:  IBM/360/67 , OS/MV T / BATCH 

and  TIME  SHARING  (CP/CMS) 

Design  notes  :  see  ANNEX  A 

Program  listing  :  see  ANNEX  B 

Coding  notes  :   see  ANNEX  C 

Debugging  notes  :   see  ANNEX  D 

Error  Listing  :    see  ANNEX  E 

Final  statistics  :   see  ANNEX  F 

Graphical  representation  :  see  ANNEX  G 

Test  chase  description:  see  ANNEX  H 


Starting  date:  19  APRIL  77 


Ending  date:  3  MAY  77 


EXPERIMENT  DESCRIPTION 
1.  Project  description: 

A.   General  Descriotion 

The  program  is  designed  for  usage  under  CP/CMS.  It 
expects  an  input  file  labeled  "DBASE  INPUT"  to  contain  data 
base  information  of  the  format  described  in  section  B. 

During  initialization  all  data  base  information  is   read 
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into  program  memory.  After  this  the  user  may  operate  upon 
the  data  using  functions  as  described  in  section  C.  All 
functions  are  input  via  terminal. 

Most  of  the  terminology  being  used  throughout  the  pro- 
ject is  non  standard.  Therefore  the  following  explanation 
is  provided  for  better  understanding: 

a)  DATA  BASE  MEMBER:   see  description  under  B.  1. 

b)  ATTRIBUTE  VALUE  PAIR:   see  descriotion  under  8.  2. 

c)  MEMBER  ID:  The  program  assigns  a  MEMBER  ID  to  each 
data  base  member  deDending  on  the  input  seguence. 
MEMBER  IDs  are  integers  (1/2,3  ...).  The  MEMBER  ID  can- 
not be  changed  by  the  user/  however/  it  may  be  used  to 
reference  any  particular  data  base  member. 

d)  ATTRIBUTE  ID:  The  program  assigns  an  ATTRIBUTE  ID 
to  each  new  and  distinct  ATTRIBUTE  NAME  being  entered 
during  the  initial  incut  of  the  data  base  members. 
ATTRIBUTE  IDs  are  integers  (1,2, ...64)  depending  also  on 
the  inout  seguence.  Each  ATTRIBUTE  ID  is  associated 
which  a  unique  ATTRIBUTE  NAME  and  provides  a  way  of 
referencing  any  particular  ATTRIBUTE  NAME  (see  section  C 
"LISTA"). 

e)  ATTRIBUTE  NAME:  An  ATTRIBUTE  NAME  is  a  character 
string  of  length  1-64  Dreceding  the  ":"  in  the  input  of 
an  ATTRIBUTE  VALUE  PAIR  (see  B ."  2.). 

f)  ATTRIBUTE  VALUE:  An  ATTRIBUTE  VALUE  is  a  character 
string   following   the   ":"   in   the   definition   of   an 
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ATTRIBUTE  VALE  PAIR  (see  B.  2.). 

g)  KEY  ATTRIBUTE:  The  KEY  ATTRIBUTE  refers  to  the 
second  distinct  attribute  name  being  input.  (The  second 
attribute  name  has  been  chosen  because  the  main  usage  of 
the  program  will  have  data  base  members  which  are  iden- 
tified uniguely  by  their  second  attribute  value  pair.) 
Therefore  the  ID  of  the  KEY  ATTRIBUTE  is  initially  2, 
however,  this  may  be  changed  by  the  user  to  any  other 
ATTRIBUTE  10  by  using  the  function  "KEY"  (see  section 
C)  . 

The  KEY  ATTRIBUTE  is  not  always  the  "key  attribute"  in 
the  sense  being  used  in  most  data  bases*  because  it  does 
not  necessarily  allow  a  unique  identification  of  all 
data  base  members.  (Instead  the  MEMBER  ID  can  be  viewed 
as  internal  key  which  always  allows  a  unique  identifica- 
tion of  each  individual  data  base  member.) 
h)  CONTROL:  If  a  data  base  member  is  taken  into  con- 
trol by  using  the  function  "CONTROL"  an  internal  refer- 
ence is  set  by  the  program  which  allows  subsequent  exam- 
ination of  ATTRIBUTE  VALUE  PAIRS  of  this  oarticular 
member  by  using  functions  "FA"  and  "ATTR". 
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B.   Input  Format : 

1.  Description  of  a  Data  Base  Member: 

A  data  base  member  is  defined  by  1  -  64  different 
attribute  value  pairs  followed  by  the  termination  symbol 
"#"  .  After  the  last  data  base  member  an  additional  ter- 
mination symbol  has  to  be  inserted. 

2.  Description  of  Attribute  Value  Pairs: 

Each  attribute  value  pair  has  the  following  format: 

<attribute  name>:<attribute  v  a  1  u  e  >  * 
where  "attribute  name"  is  a  character  string  of  length  1 
63   not  including  H:"  and  "attribute  value"  is  one  of 
the  following  types: 

a)  <single  discrete  value> 

A  single  discrete  value  is  a  character  string  of 

length   0   •  ri/  where  n  is  limited  by  the  amount 

of  memory  available  (see  section  D). 

A   character  string  of  length  0  will  be  redaced 

internally  bv  "***"   and   specifies   an   unknown 

value. 

b)  <$<multiple  discrete  values>> 

Multiple  discrete  values  are  any  number  of  sin- 
gle discrete  values  separated  by  "r".  The  total 
string  length  of  all  values  including  separating 
" / "  may  not  exceed  64. 
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c)  <o)<range  value>> 

A  range  value  is  a  string  of  length  6   -   64   of 

t  he  format : 

<1ower  limit>  TO  <uoper  1 i m i t > 
where   "lower   limit"   and  "upper  limit"  are    in- 
teger values  and  <lower  limit>  <=  <upper  limit>. 

Examo 1 es : 

a)  Attribute   value   oair   with  single  discrete 
value: 

MAME:SMITH# 

b)  Attribute  value  pair   with   multiple   dicrete 
val ues : 

CHECKING  ACCOUNT:$804020, 50033* 

c)  Attribute  value  pair  with  range  value: 

MONTHLY  PAYMENTS:3550  TO  725# 

3.  Number  of  data  base  members: 

The  data  base  may  contain  1  -150  members  depending 
upon  the  amount  of  memory  being  used  for  the  storage  of 
values  (see  section  0). 

4 .  Termination  of  inout: 

In  order  to  terminate  the  inout  of  data  base  infor- 
mation an  additional  termination  symbol  ( "  # " )  has  to  be 
inserted  after  the  the  termination  symbol  of  the  last 
data  base  member. 
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5.  Reserved  characters: 

All   characters   other   than  H : " ,  " , " ,  "U",     "$",  and 
"3"  may  be  used  without  restriction. 
6  .  Samp  1  e  i  nput  : 

NAME:MEYER#  FIRST  NAME:J0E*  AGE:57*3  NAME:SMITHtf 
FIRST   NAMElMIKE*    AGE:**  NAME:NEWMAN« 

FIRST  NAME:MARY   ANN*  AGE:    ZnttUU 
NOTE: 

a)  Between  attribute  value  pairs   any   number   of   blank 
characters  (0,1,2,...)  will  be  ignored  by  the  program. 
0)  Any  number  of  blank  characters  preceding  a  value  will 
be   ignored.    (In   the  example  the  age  of  the  last  data 
base  member  will  be  stored  as  "34".) 

c)  Any  number  of  blank  characters  greater  than  one 
within  a  value  or  attribute  name  will  be  reduced  to  a 
single  blank.  (In  the  examole  the  first  name  of  the  last 
data  base  member  will  be  stored  as  "MARY  ANN"and  the 
attribute  name  of  the  second  member  will  be  stored  as 
"FIRST  NAME"  although  it  was  input  as  "FIRST   NAME"  .) 
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C.   Function  Commands: 


1.  Functions  without  input  parameters: 


INPUT 
LISTA 


FUNCTION  DESCRIPTION 


All  attribute  names  are  list 
ATTRIBUTE  ID.  (For  usage  of 
functions  "ATTR"  and  "KEY".) 


ed  togetner  with  their 
ATTRIBUTE  ID  see 


LISTC 
LISTDBASE 


All  available  commands  are  listed. 


All  data  base  members  are  li 
their  attribute  value  pairs 
ID  of  each  data  base  member. 
ID  see  functions   "CONTROL" 


sted  printing  all  of 
together  with  MEMBER 

(For  usage  of  MEMBER 
and  "LISTM".) 


LIS1ALL 


All  data  base  members  are  li 
their  KEY  ATTRIBUTE  and  its 
MEMBER  ID.  (For  usage  of  MEM 
"CONTROL"  and  "LISTM"  .  The 
second  attribute  of  the  firs 
by  default  and  may  be  change 
function  "KEY".) 


sted  printing  only 
value  preceded  by  the 
BER  ID  see  functions 

KEY  ATTRIBUTE  is  the 
t  data  Case  member 
d  by  using  the 


SWITCH 


This  allows  the  user  to  dire 
output  from  terminal  to  a  fi 
OUTPUT"  or  or  vice  versa.  T 
from  terminal  to  file  might 
large  output  is  exDected.  T 
output  file  may  be  obtained 
"OFFLINE  PRINTCC  DBASE  OUTPU 


c  t  the  subsequent 
le  labeled  "DBASE 
o  change  the  output 
be  appropr  i  a t  e  if 
he  contents  of  the 
using  the  CMS  command 
T". 


Termination  of  the  program. 
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2.  Functions  which  require  only  one  input  parameter: 


INPUT  OF 
FUNCTION 


PARAMETER 


FUNCTION  DESCRIPTION 


LISTM 

CONTROL 


<MEMBER  I0> 
must  be  i  nt  eger 


All  attribute  value  pairs  of  data  base 
member  specified  by  MEMBER  ID  are  listed 


<MEMBER  ID> 

must  be  i  nt  ege  r 


Program  takes  data  base  member  specified 
by  MEMBER  ID  into  control.  (For  usage  of 
this  function  see  functions  "FA"  and 
"ATTR".) 


ATTR 


<ATTRIBUTE  ID> 
must  be  i  nt  eger 


List  value  of  attribute  specified  by 
ATTRIBUTE  ID  of  data  base  member  in 
control.  (To  select  proper  ATTRIBUTE  ID 
see  function  " LISTA".  To  take  a  member 
into  control  see  function  "CONTROL".) 


FA 


<ATTRIBUTE 

NAME> 
must  be  a  cha- 
racter string 
of  length  1-63 
fol lowed  by  ":" 


List  value  of  attribute  specified  by 
ATTRIBUTE  NAME  of  member  in  control. 
(To  take  a  member  into  contol  see 
function  "CONTROL".  To  select  an  existing 
ATTRIBUTE  NAME  see  function  "LISTA".) 


KEY 


<ATTRIBUTE  ID> 
must  be  i  nt  ege  r 


Change  key  attribute  of  data  base  to 
attibute  specified  by  ATTRIBUTE  ID. 
(For  usage  of  key  attribute  see  functions 
"LISTALL"  and  "FIND  ID".  To  select  an 
ATTRIBUTE  ID  see  function  "LISTA".) 


Notet  If  parameter  is  required  to  oe  an  integer  any  other 
input  will  force  the  proaram  to  terminate.  Other  wrong 
input  will  cause  an  approoriate  error  message. 
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3.  FIND  and  FIND  ID: 

a)  Function  Descrition: 


Thes 
memo 
subs 
val  u 
owns 
the 
case 
val  u 
t  i  on 
memo 
t  hos 
are 
Dai  r 
at  t  p 
att  p 
cond 
1  i  st 


e  t 

ers 
equ 
e  d 

t 
att 

w 
e  i 
s 

ers 
e 

I  is 
s 

i  bu 
i  bu 
i  t  i 
ea . 


wo  f 
wh  i 

ent 

a  i  p . 

h  i  s 

pi  bu 

i  1  1 

nth 

have 
as 

memb 

ted. 

ape 

tea 

te 

ons 


unc  t 
ch  s 
i  npu 

A  m 
papt 
tew 

be 

e  ou 

be 

to  w 

ers 

us 

pp  i 
nd  i 
of 
are 


l  ons 
at  i  s 
t  . 

embe 
i  cu  1 
i  tho 
i  no  i 
tout 
en 

heth 
wh  i  c 
i  ng 
nted 
ts  v 
the 
ent  e 


allow 
f  y  1  th 
(By  a  c 
p  satis 
ar  at  t  p 
ut  a  v 
ca t  ed  b 

follow 
input  t 
er  they 
h  satis 
"FIND" 

wherea 
a  1 ue  is 

data  b 
red  all 


the 
pu  a 
ondi 
f  i  es 
ibut 
a  1  ue 
y  pp 
i  ng 
he  p 

sat 
fy  a 
al  1 
s  us 

pr  i 
ase 

dat 


user  t 
condi 
t  i  on  i 
thee 
e  v  a  1  u 
spec 
i  nt  i  ng 
a  quer 
rogpam 
i  s  f  y  a 
1  1  con 
of  the 
ing  "F 
nted. 
see  f  u 
a   oas 


0  fi 
t  i  on 
s  me 
ondi 
e  pa 
i  f  i  e 

"** 

y.) 
tes 

1  1  c 
di  t  i 
i  p 
IND 
(To 
net  i 
e   m 


nd  al  1 
s   spe 

ant  an 
t  i  on  o 
i  r  op 
d.  T 
*  *  "  as 
After 
ts  al  1 
ond  i  t  i 
ons  of 
att  ri  b 
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b)  Input  of  Conditions: 


CONDITIONS  1-4: 


Input  of  <ATTRIBUTE  NAME> 


Input  of  Ottribute  value> 


ATTRIBUTE  NAME  can  be  any 
String  of  length  1-63 
fol 1  owed  by  H  :  "  (it  is 
necessary  that  the 
ATTRIBUTE  NAME  matches  with 
one  of  the  existing 
ATTRIBUTE  NAMES) 
If  an  ATTRIBUTE  NAME  cannot 
be  found  an  error  message 
is  ppinted  and  the  user  can 
repeat  t  he  i  nput  . 


flithin  aueries  attribute  values  are 
restricted  to  be  1-64  characters.  (If 
the  value  stored  is  longer  than  64 
characters  the  comparison  is  only 
carried  out  up  to  the  last  character 
of  the  query  value.  ) 
Since  all  values  are    stored  without 
preceding  blank  characters  the  query 
value  should  not  have  blank  characters 
in  front  of  the  value  to  be  tested  for 


c)  Termination  of  Query  Input: 

User  i  nput  s : 
Q 

d)  Sample  Input  for  FIND  and  FIND  ID: 

FIND 
NAME: 
SMITH 
AGE: 
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22 
Q 

FIND  10 
AGE: 
104 
Q 

NOTE: 

1.  All   input   for   all   functions   must  not  be  preceded  by 
blank  characters. 

2,  Each  input  has  to  be  on  a  separate  line.   The  input  is 
entered  by  hitting  the  carriage  return  key. 

0.   Limits  of  the  Program: 

The  total  number  of  different  attribute  names  may  not 
exceed  64.  Assuming  100  data  base  members  having  64  attri- 
butes each,  the  overhead  used  for  i  mD  1  emen  t  at  i  on  (t>5  items 
for  each  data  base  member)  would  reduce  the  amount  allocat- 
able  items  of  M2  (20,000)  to  13,500  items.  Each  item  of  M2 
can  be  used  to  store  1-4  characters.  The  number  of  remain- 
ing items  divided  by  the  number  of  data  base  members  gives 
the  average  number  of  items  available  for  each  data  base 
member  (  approximately  135).  Thus  the  total  amount  of 
remaining  storage  locations  allow  approximately  54,000  char- 
acters for  values  which  means  that  the  average  string  length 
of  values  will  be  less  than  8.  The  program  assumes  that  at 
least  two  different  ATTRIBUTE  NAMEs  are  entered.  If  this  is 
not  true  the  user  has  to  change  the  KEY  ATTRIBUTE  to  "  1  "  . 

E.   Error  Messages: 

All  error  messages  are  self  exDlanatory.  Error  #1  and 
#2  will  cause  the  program  to  terminate.   This   will   be   the 
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case  when  all  memory  locations  are  exhausted  (error  #1)  dur- 
ing the  input  of  data  base  information  or  more  than  1000 
members  are  specified  (error  H    2). 

Error  #3  indicates  that  an  attribute  name  is  longer  than 
63  characters.  (In  this  case  the  problem  might  be  solved  by 
examining  the  input  file.) 
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2.  Programmer's  background: 

a)  Experience  in  programming: 

Oct  1970  -  May   1971    Programming  courses 

May  1971  -  April  1972   Module  Programmer 

May  1972  -  June   197a   Work  in  Test  and  Simulation  Depart- 
ment at  the 

NAVAL  COMMAND  AND  CONTROL  SYSTEMS 
COMMAND  (FEDERAL  GERMAN  NAVY) 
Testing  of  tactical  real  time  systems 

March  1975  -  Jan  1977   Student  at  the  NAVAL  POSTGRADUATE 

SCHOOL/  Monterey*  Computer  Science 

b)  Experience  in  testing: 

Two  years  of  work  in  testing  and  simulation. 

c)  Experience  in  the  area  of  the  given  oroblem: 

Usage  of  similar  aata  structures  in  previous  programming 
pro j  ec t  s  . 

d)  Experience  in  the  oroaamming  language  being  used: 
Experience  over  a  period  of  18  months  in   more   than   10 

programming   projects.    (Total   number  of  source  statements 
produced  during  that  time  was  more  5000.) 

3.  Psychological  factors: 

a)  Did  the  programmer  like  the  project?   Yes. 
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bj  How  does  the  programmer  like  the  programming 

1 anguage? 
Favorite  programming  language. 

c)  Was  the  programmer  satisfied  by  the  way  the  problem 
was  spec  i  f i  ed? 

Only  minor  criticism. 

d)  How  aid  the  programmer  like  the  programming  environ- 
ment ? 

The  facilities  (study  room,  card  ounch  room)  were  not 
conducive  to  efficient  programming  because  of  restricted 
space*  bad  lighting  and  noise. 

e)  Other  factors: 

The  recording  of  the  experiment's  data  during  the  pro- 
ject affected  speed  and  concentration  considerably. 
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U,    Comments  on  Documentation 

For  the  documentation  of  each  software  development  phase  a 
special  documentation  form  has  been  developed.  These  forms 
are  designed  to  provide  a  firm  guideline  for  the  experiment 
programmer  to  recoro  all  data  of  interest  for  subsequent 
error  ana  lysis. 

Begin  and  end  of  each  steD  was  recorded  with  respect  to 
day  and  time. 

Each  error  was  recorded  when  it  is  discovered.  The  error 
was  then  identified  by  a  unique  error  number  (b2>  ...). 
Furthermore  the  time  of  discovery  and  the  error  type  (using 
error  types  listed  in  ANNEX  F)  were  recorded. 

-  If  appropriate/  comments  about  error  discovery/  reason  why 
the  error  was  made/  etc.  were  documented  in  ANNEX  E  2 . 

For  each  error  the  phase  in  which  the  error  was  made/  the 
phase  in  which  the  error  was  discovered  and  the  time  spent 
to  correct  the  error  was  recorded  in  ANNEX  El. 

-  For  each  steo  in  any  one  of  the  software  development 
phases  the  day/time  of  begin  and  end  was  recorded.  In  addi- 
tion/ the  time  (in  man  hours)  for  each  step  was  recorded. 
This  excludes  the  overhead  used  for  documentation  of  the 
exper  i  ment  data. 
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ANNEX  A 


Page  1   of  3 


WORKSHEET  FOR  DESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  U 


,  MAN 

1 

STEP!  PROBLEM  AND  PLANNED  SOLUTION 

ALTERNATE 

DAY 

'HOURS 

ERROR!  COMMENT 

*  i 

SOLUTIONS 

TIME 

,/STEP 

tt      ! 

1  [Analysis  of  reauirements  of 

,«/19 

! pro j  ec t . 

1U00 

5.0 

i-Since  almost  all  exoected 

Storage  in  fixed 

lvalues  are  of  different  length 

1 engt  hs  st  r i  ng 

(usage  of  linked  lists  for  sto- 

arrays  wou 1 d 

! raae  of  all  attribute  values 

was  t  e  too  much 

[seems  to  be  eooropriate. 

memory . 

!-Study  of  I/O  caoabilities  un- 

Ider  CP/CXS. 

[-View  of  data  base  members: 

[Each  data  Base  member  has  a 

iuniaue  identifier  (member  a). 

[For  each  memoer  max.  fed  diffe- 

a)  Let  numoe r  of 

rent  attribute  value  pairs  can 

d  i  f  f  eren t  attri- 

!be defined  by  incut.  Maximum 

butes  be  an  in- 

[string length  for  any  attri- 

put oa  r amet  e  r . 

bute  is  si.  The  total  number 

b)  Use  a  hash 

!of  different  attributes  in  the 

t ab 1 e  to  store 

[data  oase  may  also  not  exceed 

attributes.  This 

[64  (arbitrarily  chosen  limit). 

woulc  make 
string  lengths 
of  attributes 
indeoendent  of 
the  array  s  i  ze 
of  the  attribute 
tabl e. 

U/19 
1900 

2    {Define  data  structures: 

U/19 

a.O 

'-Attributes  arv    identified  by 

1900 

!a  character  string  of  length 

[6U  and  stored  in  a  string 

[array  (ATTRIBUTE). 

{Each  attribute  can  be  uniauely 

[identified  by  its  oosition 

[within  the  attribute  array 

[Primitives 

[(ATTRIBUTE  ID). 

[for  data 

[-Values  of  attributes  are 

!  st  rue tures 

[stored  in  a  linked  list  struc- 

! coul d  oe 

ture  using  a  BITS  ARRAY  (M2). 

!  COP  i  edf  rom 

[(Length  of  1*2  U2a,000  oytes) 

a/19 

!  pro  j  ec t 

[limits  the  amount  of  informa- 

0900 

[*2. 

tion  to  be  stored  in  the  data 

[  base . ) 

Remarks:  see  oage  la 
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ANNEX  A 


Page  la  of  3 


Remarks : 


a)  Linked  list  of  data  base  members: 


1: 


pt  p 


Dtr- 


ot  r 


pt  P' 


ptP 


NIL 


at  t  pi  but e  value 
list  of  membep  1 


at  t  p  i  but e  value 
list  of  membep  2 


attpibute  value 
list  of  membe P  n 
(last  membep ) 


b)  Structure  of  attribute  value  lists: 


Pt  r 


Al  : 


pt  P" 


Pt  r 


A2! 


PtP- 


Dtr 


A(n) 


ML 


value  st  rue  t  ure 
of  first  attpibute 


value  st  rue t ure 

of  secona  attribute 


value  st  rue  t  ure 
of  nth  attrioute 


where  Al,  A2,...  represent  the  ATTRIBUTE  IOs  of  corresponding 
values 

c )  Value  st  rue  ture: 


t  !ptr 

VALUE  REPREi 


t  lotr  — 
SENTATION..  ! 


tl  NIL 


where  t  represents  the  type  of  the  value: 

t =0:  s  i  ngl e  value 

t  =  1 :  range  va 1 ues 

t  =2:  multiple  va 1 ues 

*  is  used  as  delimiter. 

Usage  of  "***«"  for  value  representation  indicates  that  value 
is  not  known. 
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ANNEX  A 
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WORKSHEET  FOP  DESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  U 


STEP 

u 


PROBLEM  AND  PLANNED  SOLUTION 


ALTERNATE 
SOLUTIONS 


!  MAN  : 
DAY  !HOURS!ERROR 
TIMEI/STEP!  U 


COMMENT 


a) 

of 
■FI 
-GE 
-ST 
-CA 
-TY 
-FI 
-GE 
-GE 
■FI 
b) 
•SP 
•IN 
a) 
■Li 
■Li 
one 
ChO 
■Li 
al  1 
eac 
-Li 
(L 
-Li 
!  oai 
(L 
-Se 
ret 
of 
-Li 
val 
t  ro 
-Fi 
wh  i 
(F 
-Sw 
to 
-Ch 
-Ob 
-Su 
b) 
die 
!  di  n 
Remarks : 
ef  f ect  i  v 
ex  t  ens  i  o 


Def  i  ne  utilit 
data  base  i  nf 
LL,  EXTRACT, GE 
T  OLD  FILE  IN 
ORE, CLEARING 
RDIN, PRINTOUT 
PIN,TYPOUT 
LE  I  0,TERMIN 
T  ATTRIBUTES 
T  ATTRIBUTED 
ND  ATTRI8UTE 
other  utiliti 
ACE, ERROR, DIA 
ITIALIZE  ALL 
Def  i  ne  c omman 
st  all  at  t  r i  b 
st  all  member 

attribute  va 
sen  by  user) 
st  all  member 

attribute  va 
h  member  (LIS 
st  all  ava  i  1  a 
ISTC) 

st  all  at  t  r i  b 
rs  of  a  s  i  nal 
ISTM) 

1 ec  t  a  single 
rieval  of  par 
interest  (CON 
st  a  oart  i  cu I 
ue  pair  of  me 
1  (FA,ATTR) 
nd  all  oat  a  b 
ch  with  given 
INO,  FIND  ID) 
itch  output  f 
file  and  vice 
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per  user  mode 
A  1 1  sub  rout  i  n 

commands  are 
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The  last  two 
e 1 y  used  for 
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TCHAR,SKIP 
FORMATION 
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AL  I  0 

ANO  VALUES 

ET  VALUE 
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s  ( us  i  ng  only 

1 ue  oa  i  r 

(LISTALL) 

S  (orintina 

lue  Pairs  of 

T08ASE) 

ble  commands 

ut e  value 

e  member 

member  for 
ticular  values 
TROD 
ar    attribute 
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ase  members 
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rom  terminal 
versa(SWlTCH) 

ibute  (KEY) 

ics  (D) 
(SU) 

es  which  han- 
namea  accor- 


a/20 
0900 


u/20 
1300 
a/21 
0900 


functions  are  not  required,  how 
testing  purooses.  They  also  sue 
gram. 


a/21 
1100 

ever , 
cor  t 


a.O 


2.0 


they  can  be 
further 
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WORKSHEET  FOR  DESIGN  PHASE  AND  DESIGN  REVIE*  PHASE  OF  PROJECT  * 


STEP 

n 


PROBLEM  AND  PLANNED  SOLUTION 


ALTERNATE 
SOLUTIONS 


!  MAN 
DAY  JHOURS 
TIMEI/STEP 


ERROR 


COMMENT 


Ana 
(FI 
Def 

-GE 
-GE 
-GE 
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-TE 
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Def 
S  i  n 
-FI 
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-FI 
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NDATTR 
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STATTR 
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print 
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ONS 
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et e  values: 

TE 

ING 
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SULTS 
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thin  a  linked  list, 
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ed  after  all  data 
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Design  of  retrieval  functions 

for  multiple  values  and  range 

va 1 ues : 

-COMPARE  MANY 

-LLIMIT,ULIMIT 

(return  upDer  and  lower  limit) 

Design  review  and  imolementa- 
tion  of  improvements. 
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1230 


a/22 
1430 

a/23 
1530 


a/25 
1  100 

a/27 

1300 
a/27 
2030 


1  .5 


6.0 


1.5 
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Program  Listing  of  Project  U    4 
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ANNEX  C 


Page  1  of  2 


WORKSHEET  FOR  COOING  PHASE  OF  PROJECT  * 
Beginning  of  Coding  (day/time)  :  0/20/ia00 
End  of  Coding    (day/time)  :  5/02/1200 
Man  hours  :  24.5  (including  ounching  of  cards) 


CODING 
3EGIN      !     END 
DAY/TIME   !  DAY/TIME 


PROGRAM  PART 


n 

ERt>OR!DAY 
s       iTIVE 


COMMENT 


oa/20/iuoo 

0a/20/2030 
Oa/21 / 1 330 
OU/21/1530 
Oa/21/2030 

0a/21/2230 

04/22/1500 

Ou/22/1830 
Oa/25/1330 


0a/20/1800 
04/20/2330 
Oa/21/1530 
0a/21/1830 

04/21/2130 

Oa/21/2330 

04/22/1800 
04/22/1930 

04/25/1630 


Coaina  of  or  i  m  i - 
t  i  ves  and  ut  i  I  i  - 
t  i  es 

Punching  caras 


Cod  i  na  of   com" 
mand  f unc  t  i  on  s 

Punc h i  ng  caras 


Cod  i  ng  of   com* 
mand  f unc  t  ions 
and  ounching 
cards 

Codi  ng  of   com- 
mand functions 
and  ounching 
cards 

Coo  i  na  of 
retrieval 
f unc  tions 

Punch  i  ng  cares 


Coding  of  re- 
ret  r i  eva 1 
f unc t  ions  for 
mu! t ipl e  va  1 ues 
and  range  values 


1830 


1)  Record  when  error  is 

det  ec t  eo . 
A  great  number  of 
orimitives  could  be 
cooied  from  project  *2. 


C<5 
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ANNEX  C 


Page  2  of  2 


WORKSHEET  FOR  COOING  PHASE  OF  PROJECT  * 
Beginning  of  Coding  (day/time)  :  U/20/1400 
End  of  Coding    (day/time)  :  5/02/1200 
Man  hours  :  24.5  (including  punching  of  cards) 


COOING 


8EGIN 
OAY/TIWE 


ENO 
0  AY /TIME 


PROGRAM  PART 


1) 
ERROPiDAY 
«       [TIME 


C0MMENT 


04/25/2100 


04/26/1530 


04/27/2030 


05/02/1 100 


04/25/2200 


04/26/ 1630 


04/27/2100 


05/02/1200 


Punch  ina  cares 


Coding  of 
debugging  aias 
and  ounch  i  ng 
cards 

Changes  accor- 
ding to  design 
rev  i  ew 

Coding  of  new 
Drocedures  to 
hand  1 e  range 
value  comoarison 
and  ounc h  i  ng 
cards 


35 

36 


48 


2100 
2200 


2050 


1)  Record  when  error  is 
det  ec  t ea  . 

Al 
Al 


C17 


Change  due  to  error  tt    50, 
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ANNEX  0 


Page  1  of  10 


WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  *  :  a  DEBUG  Run  *  :  1 

Begin  of  Debug  Run  (day/time)  :   04/20/2330 

End  of  Debug  Run    (day/time)  :   04/21/2030 

*  of  Debug  Steos  incl.  in   Debug  Run:  4    CPU  time  for  Debug  Pun  (sec):  0.0 

CPU  time  for  necessary   comoiles  (sec)  :  19.39 

a)   3.93    b)   a. 42    c)   5.34    a)  5.70    e)  f)         g) 

Van    nours  for  this  Debug  Run  :  2.5  (including  Dreoaration  of  debug  run) 

MAN        1  ) 


STEPJPROGRAM 

*  !   PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY IHOURS [ERROR 
TIMES/STEP!   * 


COMMENTS 
AND  CODED 
ERROR  TYPES 


Primi- 

tives 

and 

ut  i  1  i  - 

t  i  es 

Pr  i  m  i  - 

t  i  ves# 

utili- 

ties* 

i  nput 

f unc  t- 

l  ons 

Get  error  free  comoile 


Get  error  free    comoile 


repeat  step  2 


repeat  steo  2 


5  como  i 1 e 
errors 


1  c  omo  i 
error 


1  comoi 1 e 
error 


O.K. 


J4/20 

12330 

i  .o  :     : 

14/21 

:  1030 

:  i  : 

:  2  : 

:  3  i 

14/21 

i    a    ! 

1110  0 

:  5  i 

!4/2t 

!  1630 

0.5  !      ! 

:  1900 

!   7   ! 

J4/21 

!  1930 

14/21 

:  1930 

0.5  :      ! 

12000 

!   8   : 

14/21 

12000 

!4/21 

J2000 

0.5  !      J 

J4/21 

!2030 

1  )  Record  when 
error 
occurs 

Cb 
Al 

Al 
Al 
Al 


C17 


C12 


-  257  - 


ANNEX  0 


Page  2  of  10 


WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  U    :  4  OEBUG  Run  *  :  2 

8egin  of  Debug  Run  (day/time)  :   04/21/2130 

End  of  Debug  Run    (day/time)  :   0*4/22/1830 

*  of  Debug  Steps  incl.  in   Debug  Run:  6    CPU  time  for  Debug  run  (sec):  11.83 

CPU  time  for  necessary   compiles  (sec)  :  39.55 

a)   6.48    b)   5.52    c)   5.68    a)  4.47    e)  8.26     f)  9.14    g) 

Man  hours  for  this  Deoua  Run  :  5.5  (including  preparation  of  decug  run) 

man       1 ) 


STEP'.PROGRAM 
*  !   PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY  JH0URS1ERR0R 

timej/step:     * 


COMMENTS 
AND  CODED 

ERROR  TYPES 


A|  1 

exceot 
ret  r i  e- 
val 

f unc t- 
i  ons 


Al  1 

except 

ret  r i  e* 

val 

f unc t- 

i  ons 


Get  error  free    compile 


repeat  steo  1  and  check 
initial ization 


repeat  step  2 


Get  error  free  compile 
(including  command 
f unc  t  i  ons ) 


repeat  step  4  ana    check 
initialization 


2  comoi 1 e 
errors 


1  program 
error 


5  program 
errors 


2  compi 1 e 
errors 


2  program 
errors 


repeat  step  5 


O.K. 


J4/21 

12130 

0.5  !      i 

12130 

!      1   q 

i  10 

54/21 

12200 

!  a/21 

12200 

,  0.5  1      ! 

12200 

>      1  11   1 

14/21 

12230 

14/21 

12330 

2.5  !      ! 

la/22 

:  iooo 

1  12   ! 

:  1020 

1  13   1 

:  1030 

1  14   i 

!  1  100 

1  15   1 

:  mo 

1  le   1 

14/22 

:  1200 

14/22 

:  1200 

0.5  1      ! 

:  1200 

:   17     i 

!  1230 

!  18   1 

14/22 

!  1230 

14/22 

!  1400 

1.0!      ! 

114  3  0 

119   J 

14/22 

1  1500 

1  20   1 

14/22 

1 1800' 

0.5  1      ! 

1  )  Record  when 

error 

occurs 
C7 
Al 


07 


C28 
C21 
C21 

C28 
012 


C6 
C17 


012 
C27 
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ANNEX  0 


Page  3  of  10 


WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  u    :  a  DEBUG  Run  *  :  3 

8egin  of  Debug  Run  (day/time)  :   04/22/1830 

End  of  Debug  Run    (day/time)  :   Oa/22/2200 

*  of  Debug  Steps  incl.  in   Debug  Run:  2    CPU  time  for  Debug  run  (sec):  11.93 

CPU  time  for  necessary   compiles  (sec)  :  17.75 

a)   8.71    b)   9.oa    c)         a)        e)  f)        g) 

Man  hours  for  t n i s  Debug  Run  :  2.5  (including  oreoaration  of  flebuq  run) 

M an       i  ) 


STEP 

9 


PROGRAM 
PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY  :  HOURS !ERR0R 
TIME'./STEP!   « 


COMMENTS 
AND  CODED 
ERROR  TYPES 


Al  1 

exceot 
f unc t  - 
t  i  ons 
to  com- 
pare 
range 
val ues 
or  mu 1  - 
tide 
di  s- 
c  ret  e 
va  I  ues 


Get  error  free    comoile 


3  comoi 
errors 


repeat  step  1  and  examine 
storage  of  data  base  in- 
f  o  rma t  i  on 
(all  attributes  and  values 
should  be  imolementeo 
according  to  aesign) 


0.K 


a/22 
1930 
I9a5 
1950 
1^55 


a/22 
2000 

a/22 

2000 


a/22 
2200 


0.5 


21 
23 


1  )  Record  »hen 
error 
occurs 


Al 
A2 
C23 
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ANNEX  0 


Page  a  of  10 


WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  0  :  a  DEBUG  Run  n    :  a 

Begin  of  Debug  Run  (day/time)  :   04/22/2200 

End  of  Debug  Run    (flay/time)  :   04/25/1100 

»  of  Debug  Steps  incl.  in   Debug  Run:  6    CPU  time  for  Debug  run  (sec):  43.47 

CPU  time  for  necessary   compiles  (sec)  :  73.14 

a)  11.36    b)   9.73    c)   9.84    d)  9.9&    e)l0.87     f)10.49    g)  10.89 

Man  hours  for  this  Debug  Run  :  7.5  (including  oreoaration  of  aebug  run) 

MAN        1  ) 


STEPiPROGRAM 
tt     ',       PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY  |H0URS!EPR0R 

time:/step!  * 


COMMENTS 
AND  CODED 
ERROR  TYPES 


Al  1 

exceot 
compa- 
r  i  son 
of  mu 1 ■ 
t  iol  e 
va 1 ues 
or 

range 
va  1  ues 


Check  basic  functions 
(exceot  FIND  function) 


reoeat  steo  1 


repeat  step  1 


repeat  steo  1 


repeat  step  1 


repeat  steo  1 


1  program 
error 


1  proaram 
error 


1  program 
error 


1  program 
error 


2  program 
errors 


O.K. 


!4/22 

!2200 

o.5  : 

12200 

!  24   ! 

\*/2l 

!2230 

!4/22 

J2230 

3.0  | 

14/23 

!  1230 

!  25 

J4/23 

!  1230 

1.0!      ! 

!  1300 

!  2b   ! 

14/23 

!  1330 

!4/23 

!  1330 

1  .0  :      ! 

114  0  0 

:  27     : 

!4/23 

:  t43o 

!4/23 

:  1430 

i.o  !    : 

!  1500 

i  28     : 

:  1500 

!  29  : 

14/23 

!  1530 

!  4/24 

:  1700 

i .o  :    i 

14/25 

11100 

1)  Recora  when 

error 

occurs 
D12 


03 
D13 

D15 


D12 
C27 
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WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  *  :  a  0E8UG  Run  *  :  5 

Begin  of  Debug  Run  (day/time)  :   04/25/1630 

End  of  Debug  Run    (day/time)  :   04/25/2400 

*  of  Debug  Steps  incl.  in   Debug  Run:  5    CPU  time  for  Debug  run  (sec):  8.95 

CPU  time  for  necessary   comoiles  (sec)  :  51.3 

a)    11.02        b)    11.48         c)     11.90         a)    9.37         e)    7.53  f)  g) 

Man  hours  for  this  Debug  Run  :  5.5  (includina  preoaration  of  debug  run) 

MAN        1) 


STEPJPROGRAM 
*  !   PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY  J  HOURS ! ERROR 
TIMEj/STEP!   a 


COMMENTS 
ANO  CODED 
ERROR  TYPES 


Find 
f unc t - 
t  i  on 
(single 
di  s- 
c  ret  e 
va 1 ues ) 


Check  FIND  function  for 
trivial  cases 
Check  inout  of  conditions 
(0-4  conditions  should  oe 
stacked  according  to 
des  i  gn ) 


reoeat  step  I 
( us  i  ng  trace) 


reoeat  steo  I 


reoeat  steo  1 


repeat   step  1 


3  program 
errors 


1  program 
error 


1  orogram 
error 


1  program 
error 


O.K. 


|u/25 

:  io3o 

l  l.o  j     ; 

:  1630 

!      1  30   | 

!  1700 

!      1  31   1 

!  1730 

!      1  32   1 

14/25 

!  1730 

14/25 

■'1700 

2.0  i              i 

12030 

1  33   1 

14/25 

12030 

14/25 

12030 

0.5  1      ! 

12100 

1  34    1 

',4/25 

J  21  00 

14/25 

12200 

0.5  1      1 

12230 

1  37   1 

14/25 

12245 

14/25 

12245 

1.5  1      | 

14/25 

12400 

1 )  Rec o ra  »nen 

error 

occurs 
it 
C27 
C12 


84  (Error  332 
not  corrected 
proper  I y ) 


C28 
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WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  *  :  tt  0E8UG  Run  »    :  6 

Begin  of  Debug  Run  (day/time)  :   0a/25/2U00 

End  of  Debug  Run    (day/time)  :   0a/26/1930 

*  of  Debug  Steps  incl.  in   Debug  Run:  7    CPU  time  for  Debug  run  (sec):  10.23 

CPU  time  for  necessary   compiles  (sec)  :  67.8a 

a)       6.57         b)     12.88         c)     12.66         a)    6.66         e)    8.9b  010.27         g)    9.8a 

Man  hours  for  this  Debug  Run  :  8.0  (including  preparation  of    debug  run) 

u  A  N       1  ) 


3TEPJPR0GRAM 

*  !   PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


day ;hours:error 
time:/step:  « 


COMMENTS 
AND  CODED 
ERROR  TYPES 


Al  1 


Get  error  tree    comoile 
and  test  FIND  function 
for  trivial  cases 
(all  members  listed  must 
match  conditions  soec i  - 
f  i  ed  by  i  nout ) 


repeat  steo  1 


repeat  steo  I 


reoeat  steo  1 
(including  deougginq 
aids) 

reoeat  step  a 


repeat  step  a 


repeat  steo  a 


!  1  )  Record 

*hen 

3  comoi 1 e 

!U/25 

!    error 

errors 

2a00 
la/lb 

2.0 

!    occurs 

:  o<?oo 

:   38 

!  Al 

10910 

!  39 

!  Al 

:o9i5 

!  ao 

:  ai 

!u/2b 

!  1000 

2  program 

!a/26 

errors 

,  1000 

2.0 

!  1050 

!  ai 

C28 

1  100 

!  a2 

,C28 

a/2b 

1200 

1  program 

a/26 

1.0 

error 

1330 
a/26 
laao 

a3 

Al 

1  comoi 1 e 

a/26 

error 

1630 
a/26 

1  .0 

1730 

aa 

Al 

1  program 

a/26 

error 

1730 

0.5 

17a5 

a5 

A3 

a/26 

1800 

1  proqram 

a/26 

error 

1800 

0.5 

1815 

a6 

Al 

a/26 

1830 

O.K. 

a/26 
1830, 

1.0 
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WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  it    :  a  DEBUG  Run  tt    :  7 

Begin  of  Debug  Run  (day/time)  :   0*4/26/2000 

End  of  Debug  Run    (day/time)  :   04/26/2300 

tt    of  Debug  Steps  incl.  in   Debug  Run:  2    CPU  time  for  Debug  run  (sec):  7.0 

CPU  time  for  necessary   comoiles  (sec)  :  18.14 

a)   9.73    b)   8.41    c)  d)         e)  f)         g) 

Man  hours  for  this  Debug  Run  :  3.0  (including  preoaration  of  deoug  run) 

*AN       l  ) 


STEP 

tf 


PROGRAM 
PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 
RESULT 


DAY  |  HOURS IERR0R 
TIMEJ/STEP!  4 


COMMENTS 
AND  CODED 
ERROR  TYPES 


Al  1 


Test  FIND  function  for 
various  inout  (incluoinq 
also  extreme  conditions) 
(all  queries  should  be 
answered  according 
aes i  gn ) 

repeat  step  ! 


1  program 
error 


O.K. 


4/26 
2000 
2120 

4/26 
2200 

4/26 

2200 
4/26 
2300 


2.0 


1.0 


47 


1 )  Record  when 
error 
occurs 


C28 
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ANNEX  D 
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WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  tf  :  a  DEBUG  Run  *  :  8 

Begin  of  Debug  Run  (day/time)  :   04/27/1500 

End  of  Debug  Run    (day/time)  :   04/27/1900 

tt    of  Debug  Steos  incl.  in   Debug  Run:  1    CPU  time  for  Debug  run  (sec):  11.98 

CPU  time  for  necessary   comoiles  (sec)  :  10.02 

a)  10.02    b)  c)  a)  e)  f)         g) 

Man  hours  for  this  Debug  Run  :  3.0  (including  creoaration  of  debug  run) 

VAN        1  ) 


5TEPJPR0GRAM 

08JECTIVE  AND    EXPECTED 

ACTUAL 

DAY 

HOURS 

ERROR 

COMMENTS 

tt     !   PART 

RESULT 

RESULT 

TIME 

/STEP 

s 

AND  COOED 

1        , 

ERROR  TYPES 

1  )  Record  when 

error 

occurs 

I   JAM 

Repeat  Debua  Run  37  under 

CP/CMS 

(same  results  exoected) 

O.K  . 

a/27 

1500 
a/27 
1900 

3.0 
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ANNEX  0 
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WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  ft  :  a  DEBUG  Run  ft  :  9 

Begin  of  Debug  Run  (day/time)  :   Oa/28/1400 

End  of  Debug  Run    (day/time)  :   04/28/1600 

ft  of  Debug  Steps  incl.  in   Debug  Run:  1    CPU  time  for  Debug  run  (sec):  19.58 

CPU  time  for  necessary   co-noiles  (sec)  :  9.«& 

a)  P. 46     b)  c)  c)         e)  f)         g) 

^an  hours  for  this  Deoug  Pun  :  2.0  (including  oreoaration  of  debug  run) 

wan        1  ) 


STEPIPROGRAM 
ft  !   PART 


OBJECTIVE  AND    EXPECTED 
RESULT 


ACTUAL 

RESULT 


DAY  iHOURS  iERROR 

timei/step:  3 


COMMENTS 

AND  CODED 
ERROR  TrPES 


Al  1 


Var  i 

CP/C 
•Test 
■Test 

at  t  r 
■Test 

avai 
■Test 

al  1 

i  npu 
■Chec 

of  u 
■Test 

i  1  le 
■Test 

AHR 

i  n  c 
■Test 

out 
■Test 

at  t  r 

user 
■Test 

bers 

al  1 

pai  r 
■Test 

1  i  st 

bute 
■Test 

var  i 

nat  i 


s  tests  under 

nitial ization 
ISTA  (list  of  all 
utes  ) 

ISTC  (list  all 
b 1 e  commands ) 
ISTM  (check  for 
moer s  and  i 1 1 ega 1 

program  for  input 
efined  commands 
ONTROL  (check  for 
1  i  nout  also) 
unc  t  i  ons  FA  ana 
or  several  members 
t  rol 

»*ITCH  (examine  out- 
let 

EY  (change  of  key 
ut e  as  aes i rea    by 

ISTDBASE  (all  mem- 
re  1 i  sted  with  all 
tribute  va 1 ue 

ISTALL  (members  are 
on  1 y  by  kev  at  t  r i - 

IND  function  (using 

s  i  nout  como  i - 

s) 


C.K, 


a/28 

laoo 


a/28 
loOO 


2.0 


1)  Recora  when 
error 
occurs 
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ANNEX  D 
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WORKSHEET  FOR  DEBUGGING  PHASE 

PROJECT  ft  :  a  DEBUG  Run  *  :  10 

Begin  of  Debug  Run  (day/time)  :   OU/28/1600 

End  of  Debug  Run    (day/time)  :   Oa/28/1800 

*  of  Debug  Steps  incl.  in   Debug  Run:  1    CPU  time  for  Debug  run  (sec):  12.08 

CPU  time  for  necessary   comoiles  (sec)  :  P.a6 

a)  9. Hb  b)  c)  a)         e)  f)         g) 

rian    hours  for  this  Debug  Run  :  2.0  (including  preparation  of  debug  run) 

HAN       I) 


STEP 

PROGRAM 

OBJECTIVE  AND    EXPECTED 

ACTUAL 

DAY 

HOURS 

ERROR 

COMMENTS 

* 

PART 

RESULT 

RESULT 

TIME 

/STEP 

* 

AND  CODED 
ERROR  TYPES 

1 )  Reco rd  when 

error 

occurs 

1 

Al  1 

Various 
CP/CMS: 
a)  Test 

tests  under 
SU  (super  user 

1  o  roa  ram 

a/28 
1600 

2.0 

f  unc  t 

i  on  ) 

error 

162  0 

U9 

C27 

b)  Test 

FIND  function 

for  0  conditions  (all 

O.K. 

members 

shoul d  oe  1 i  s t  ed) 

c)  Test 

FIND  <amd  FIND  ID 

using  I 

-4  different  con- 

O.K. 

di  t  i  ons 

(check  correc  t - 

ness  of 

Drogram  especially 

mu 1 t  i  o 1 e 

va 1 ues  and  range 

va 1 ues ) 

/ 

d )  Test 
i 1 1 ega 1 
shou 1 d 
pr i  at e 

*E Y  f unc  t  i  on  for 
i  nou t  s  (proar am 
provide  an  appro" 
error  message) 

O.K. 

a/28 
18  50 
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ERROR  LISTING 
PROJECT  «  :  4 

Begin  of  Project  (day/time)  :  04/1R/1U00 
End  of  Project  (day/time)  :  Oa/03/1800 
Man  hours  for  total  project  :  101.0 


ERROR 

PHASE 

PHASE 

ERROR 

TIME 

*  of  OTHER 

* 

in  which 

in  which 

TYPE 

soent  to 

STATEMENTS  OR 

ERROR  was 

ERROR  was 

(see  ANNEX  F) 

solve  the 

PARTS  Or  THE 

di  s- 

iiade 

ERP0R 

PROGRAM 

covered 

("an 

t\  i  n  .  ) 

AFFECTED 

1 

Debuggi  ng 

Cod i  ng 

Cto 

5 

2 

Debuggi  ng 

Codi  ng 

Al 

i     5 

3 

Debugq  i  ng 

Codi  ng 

Al 

!     5 

a 

OeOuggi  ng 

Codi  ng 

Al 

!     5 

5 

Debuggi  ng 

Cod i ng 

Al 

!     5 

6 

Codi  nq 

Coai  na 

C9 

!     5 

7 

Debugg  i  ng 

Codi  ng 

C17 

!     5 

3 

Oeouggi  ng 

Cod  i  ng 

C12 

5 

9 

Debuggi  ng 

Codi  ng 

C7 

5 

10 

Debuggi  ng 

Cod  i  ng 

Al 

5 

1  1 

Deouggi  ng 

Des  i  gn 

D7 

5 

12 

Debuggi  ng 

Codi  ng 

C28 

5 

13 

Debuggi  ng 

Cod  i  ng 

C21 

15 

14 

Debuggi  ng 

Cod  i  ng 

C21 

!    15 

15 

Debuggi  nq 

r  Codi  ng 

C28 

!    10 

16 

Debuggi  ng 

Des  i  gn 

012 

!    25 

17 

Debuggi  ng 

Codi  ng 

C6 

!     5 

18 

Debuggi  ng 

Codi  ng 

C17 

|     5 

19 

Debugg  i  ng 

Des  i  gn 

D12 

30 

20 

Debuggi  ng 

Codi  ng 

C27 

10 

21 

Debuggi  ng 

Coding 

Al 

5 

22 

Debugqi  ng 

Codi  ng 

A2 

5 

23 

Debuqq  i  ng 

Cod  i  ng 

C23 

5 

2a 

Debugg i ng 

Des  i  qn 

012 

10 

25 

Debuggi  ng 

Des i gn 

03 

15 

26 

Debuggi  ng 

Des  i  an 

013 

30 

27 

Debuggi  ng 

Desi  qn 

015 

20 

28 

Debuggi  ng 

Desi  gn 

D12 

30 

29 

Debuggi  ng 

Codi  ng 

C27             ! 

5     | 

30 

Debuggi  ng 

Codi  ng 

Al               ! 

5     ! 
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ANNEX  El 
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ERROR  LISTING 
PROJECT  n    :    a 

8egin  of  Project  (day/time)  :  04/19/1400 
End  of  Project  (day/time)  :  OU/03/1800 
Man  hours  for  total  project  :  101.0 


ERROR 

n 


PHASE 

in  which 

ERROR  -as 

di  s- 

covered 


PHASE 
in  which 
ERROR  *as 
made 


EPROR 

TYPE 

(see  ANNEX  F) 


TIME 
soent  t  o 
solve  the 

ERROR 

mi  n  .  ) 


*  of  OTHER 
STATEMENTS  OR 

PARTS  OF  THE 

PROGRAM 

AFFECTED 


31 
32 
33 

3U 
35 
36 
37 
38 
3° 
U0 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 


Oeougai  ng 
Oebuggi  ng 
Oeouggi  ng 
Oebugqi  ng 

Cod  i  ng 

Coding 
Debuggi  ng 
Oebuggi  ng 
Debugg  i  ng 
Oebuggi  ng 
Oebuggi  ng 
Oebugqi  ng 
Oeouggi  ng 
Oebuggi  ng 
Debuggi  ng 
Oebuggi  ng 
Debuggi  ng 

Cod  i  ng 
Debuggi  ng 

Test  i  ng 


Codi  ng 
Des  i  an 

Deougai  ng 
Coai  ng 
Coding 
Codi  ng 

Deouga i ng 
Cod  i  ng 
Cod  i  ng 
Cod  i  ng 
Codi  ng 
Codi  ng 
Cod  i  ng 
Codi  ng 

DeOuggi  ng 
Codi  ng 
Codi  ng 
Codi  ng 
Codi  ng 
Oes  i  gn 


C27 
012 

84 

C28 

Al 

Al 

Al 

Al 

Al 

Al 

C28 

C28 

Al 

Al 

A3 

Al 

C28 

C17 

C27 

03 


10 

90 

15 

15 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

10 

10 

5 

30 

120 
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ERROR  LISTING  (COMMENTS) 


ERROR 

DAY 

COMMENTS 

tt 

TIME 

(EVIDENCE,  THOUGHTS,  WHY  WAS  THE  ERROR  MADE? 
,   WHY  AND  HOW  WAS  THE  ERROR  DISCOVERED? 
,   ERROR  BLOCKING,  etc.) 

04/21 

1 

1030 

Errors  *1  thru  5  are    due  to  influence  of  fatigue. 

2 

1030 

3 

1030 

a 

1030 

5 

1030 

b 

1830 

Found  while  checking  previously  written  code  during 
ounching  of  cards. 

7 

1900 

Different  variaole  name  assumed. 

8 

2000 

Missing  mandatory  declaration. 

9 

2130 

10 

2130 

1  1 

2200 
04/22 

12 

1000 

13 

1020 

1« 

1030 

Errors  #14  thru  lb  are  oue  to  insufficient  des*  checking. 

15 

1  100 

16 

1110 

17 

1200 

13 

1230 

1° 

1430 

20 

1500 

21 

1045 

22 

1950 

Uooer  case  Key  oressea  while  ounching  caras. 

23 

1955 

Usage  of  "THEN"  instead  of  "DO". (wrong  format) 

24 

2200 
04/23 

25 

1230 

26 

1300 

27 

1400 

28 

1500 

2" 

1500 
04/25 

30 

1630 
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ANNEX  E2 
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ERROR  LISTING  (COMMENTS) 


ERROR 


DAY 
TIME 


COMMENTS 

(EVIDENCE,  THOUGHTS,  wHY  WAS  THE  ERROR  MADE? 

WHY  AND  HOW  WAS  THE  ERROR  DISCOVERED? 

ERROR  BLOCKING,  etc.) 


31 
32 

33 


48 
49 
50 


04/25 
1700 
1730 

2030 


34 

2100 

35 

2100 

36 

2200 

37 

2230 

04/26 

38 

0900 

39 

0910 

40 

0915 

41 

1050 

42 

1100 

43 

1330 

44 

1730 

45 

1745 

46 

1815 

47 

2120 

04/27 
2050 

04/28 
1620 

05/01 
1700 


No  orooer  desk  checking.  (Error  could  have  been 

avoided  by  des*  checking.) 

Inaporoor i at e  correction  of  error  «32.  Debugging 

results  were  not  interoreted  correctly. 

Forgotten  "  *  1  "  . 

Errors  »35  and  36  are    due  to  f a t i gue . ( t r i v i a  1  errors) 


Errors  *38  thru  40  are  due  to  lack  of  concentration 
while  Punching  cards. 


Doub 1 e  negat  i  on 
could  have  oeen 
The  error  could 
a  desk  test. 


in  connection  with  logical  AND 
testea  bv  usage  of  a  trutn  taole. 
have  been  avoided  this  way  during 


Found  while  punching  cards. 


It  was  surDrising  to  the  exoeriment  orogrammer 
that  using  the  EBCDIC  character  set   the 
following  comparison  will  be  evaluated  "TRUE": 

Error  *  50  was  made  because  this  relationshio 
was  not  known. 
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ANNEX  F 


FINAL  STATISTICS 


PROJECT  a   a 

FINAL  STATISTICS 
Project  name  :  DATA  RETRIEVAL  SYSTEM 

Short  description: 

The  DPogra1"  is  designed  for  usage  under  CP/CMS.  It 
exoects  an  inout  file  laceled  "DBASE  INPUT"  to  contain  data 
base  information  in  a  particular  format  sucn  that  the  oro- 
gram  can  read  all  information  and  store  it  into  memory. 
After  all  data  base  members  ar^  defined  by  incut  ana  imple- 
mented within  linked  lists  the  user  mav  operate  uoon  the 
data  oase  using  functions  from  a  previously  aefinea  set  of 
f unc t  ions. 

The  program  has  been  designed  and  implemented  in  such  a 
manner  that  it  will  oe  easy  to  imolement  more  functions  or 
to  extent  the  currently  defined  limits  of  the  program. 

Quantitative  measures: 

1.  *  of  source  statements  :  1084 

2.  Total  man  hours  for  oroject  :  101.0 

3.  Man  hours  soent  in 

a)  Oesign  :   24.0 

b)  Coding  :   24.5 

c  )  Oeouggi  ng  :  ai .5 
d)  Test  ing  :  11.0 
a.  CPU  time  for  compiles:  343.57  sec. 

5.  CPU  time  for  debug  runs:  14C.05  sec. 

6.  CPU  time  for  test  runs:  33.04  sec. 

7.  a  of  test  and  debug  runs:  13 

8.  *    of  test  and  debug  steps:  43 

9 .  M    of  errors  found:  50 

10.  Total  man  hours  used  to  correct  errors:  11.25 
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ANNEX  F 


FINAL  STATISTICS 


11.  Error  Detect  i  on : 

Mean  time  between  error  detections:  87.7  man  min, 


tt    of 

errors 

t 


35 


30 


25 


20 


15 


10 


*  *  * ** 

*  *  *  *  * 

*  *  *  *  *  * 
*****        *** 

*  *  *  *  *****    *  *    ***  ***  ***  man  hours 
--♦-  +  -  +  -  +  -  +  -  +  -  +  -  +  -  +  -  +  -  +  -  +  -  +  -  +  -  +  -+'-+'-t  -  +  -♦-  +  -  +  -  +  -  +  -  +  -  +  -  +  -  +  -  +  -->  of 

1    2    3    4    5    6    7    8    9   10   11   12   13         project  time 
TIME  BETWEEN  ERROR  DETECTIONS 
(measured  from  point  in  time  of  detection  of  previous  error) 
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ANNEX  F 


PINAL  STATISTICS 


12.  Error  Correction: 

Mean  time  to  correct  an    error:  13.3  man  min. 

n    of 
Errors 
corrected . 

T 


30  + 


25 


20 


15 


10 


1 

-  + 
1 


*  *     *  * 

*  *     *  * 

*  *     *  * 
******  »         * 

5    10    15    20    25    30  90       120        180 

man  minutes 
of  dpo j  ec t  t  i  me 
TIME  TO  CORRECT  ERRORS 
(measured  from  point  in  time  of  detection) 
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ANNEX  F 


FINAL  STATISTICS 


13.  When  errors  were  founo! 


a) 
b) 
c) 
d) 
e) 


of  errors  found  during  design  ohase: 
of  errors  found  during  design  review 
of  errors  found  during  coding: 
of  errors  fo-und  during  debugging: 
of  errors  found  during  writing  of 
test  procedures: 


0 

=   0.0 

X 

0 

=   0.0 

X 

Q 

=   8.0 

as 

=  90.0 

X 

0.0  % 


f)  3  of  errors  found  during  testing: 


50 

1 


2.0  X 


14. 

When 

errors 

were 

a) 

3  Of 

errors 

made 

b) 

3  Of 

errors 

maae 

c) 

s  of 

errors 

made 

d) 

a  of 

errors 

made 

e) 

«  of 

errors 

made 

f)  s  of  errors  made 


during  lesion  ohase: 
during  design  review: 
dur i  ng  cod  i  ng : 
durina  deouagina: 
during  writina  of 
test  orocedures : 
durina  testing: 


10 
0 

37 
3 

=  20.0 

=  0.0 
=  7u.  0 
=   6.0 

X 

- 

X 

0 
0 

=  0.0 
=   0.0 

50 
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ANNEX  F 


FINAL  STATISTICS 


15.  TIME   HISTORY  GRAPHS  : 


a    of 
Errors 
t 


5* 


♦-+-♦-< 

2 


A 

A  C 

A  DCC    C   0 

A  CACCC   A   OC 

C  CCCCDO   AD  DO 
.+—+-+-+-+-+-+-+—+«+■ 

20  UO 


Error  Tyces : 

A: 

Clerical  error 

8: 

Debugging  error 

C: 

Cod  i  nq  error 

0: 

Des  i  gn  error 

A 

A 

AA 

C 

AC 

A 

A08AACAA  C   CC  D 
+  -  +  -  +  -  +  -  +  -♦-  +  -  +  -  +  -♦-♦-•»•• 
60  80         100 


•  +—+«•+->   man 

hours 


NUMBER  OF  ERRORS  FOUND  VS  PROJECT  TIME 


8    of 
Errors 


10* 


5  + 


*  **  *         *         * 

*  **  *         *       *  * 
******    **    ** 


*****    • 
*•*•**•• 


20 


■  +  -♦-♦-  +  -♦-♦-+■-♦-•♦•-  +  • 


U0 


60 


■  +  -+■-  +  -  +  -  +  • 
80 


100 


i\an 
hours 


NUMBER  OF  ERRORS  C0RPECTE0  VS  PROJECT  TIME 
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ANNEX  F 


ERROR  CATEGORIES  AND  TYPES 


1 .   Oes  ign  Errors 

The  follwing  types  of  errors  aooly   to   both   categories 
"System  Design  Errors"  and  "Program  Design  Errors": 


Dl 

02 
D3 

Da 

OS 

06 

D7 

08 

09 

D10 

Oil 

D12 

D13 

014 

015 

016 

017 


Communication  Error 

Design  Negliaence 

Forgotten  Cases  or  Steps 

Timing  Prob 1  ems 

Errors  in  I/O  Concents 

Data  Des  i  gn  Error 

Initialization  Error 

Inadeouate  Checking 

Extreme  Conditions  Neglected 

Seouenc  i  ng  Error 

Index  i  ng  Error 

Looo  Control  Errors 

Misuse  of  Boolean  Exoression 

Mathematical  Error 

Reo resen t a t i on  Error 

Misunderstanding  of  Problem  Soec i f i cat i ons 

Other  Design  Errors 


2 .   Coding  Errors 


CI 

C2 

C3 

CU 

C5 

C6 

C7 

C8 

C9 

C10 

Cil 

C12 

C13 

C14 

C15 

C16 

C17 

C18 

C19 

C20 

C21 

C22 

C23 

C2U 

C25 

C26 


Mi  su 
Negl 
1/0 
Mi  so 
MU1  t 

Miss 

Inad 
Ini  t 
Er  ro 
Inad 
Leve 
Miss 
Case 
GO  r 

Comm 
Forg 
Inco 
Wron 
Inde 
Inco 
Seau 
Flag 
Synt 
Looo 
Inco 
Lang 


nderst andi ng  of  Design 

i  qence 

Format  Error 

laced  Data  Declaration 

io'e  Data  Declarations 

i ng  Data  Declaration 

equa t e  Data 

ialization  Error 

r  in  Parameter  Passing 

equate  or  Forgotten  Checking 

1  Prob 1  ems 

ing  Declarations  of  31ock  Limits 

se  1  ec  t  i  on  error 
0  Problems 
en t  Error 
otten  Delimiter 
nsistency  in  Naming 
g  Use  of  Nested  IF  Statements 
x  i  ng  Error 
nsistent  Use  of 
enc  i  ng  Error 

Usage  Prob I  ems 
ax  Error 

Control  Error 
r  rec  t  Exit  f  rom 
uaqe  U 


Var i  ab 1 es  or  Data 


Subrout  i  nes 


saae  P roo 1  ems 
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ANNEX  F 


ERROR  CATEGORIES  AND  TYPES 


C27:  Forgotten  Statements 

C28:  Representation  Error 

C29:  Control  Sequence  Error 

C30:  Incorrect  Subroutine  Usage 

C31:  Other  Coding  Errors 


3.   Clerical  Errors 


Al 

A2 
A3 

A4 


Manua 1  Error 
Went  a  1  Error 
Procedural  Errors 
Other  Clerical  Errors 


1.   Debugging  Errors 


Bl 
82 

83 
84 
85 


Inaooropr i at e  Use  of  Debugging  Tools 

Insufficient  or  Inaooroor i at e  Selection 

of  Test  Cases  or  Test  Data 

Misinterpretation  of  Debugging  Results 

^i s i n t eror e t a t i on  of  Error  Source 

Neg 1 i  gence 

Other  Deouqgina  Errors 


5.   Test  i  ng  Errors 


Tl 
T2 
T3 
T4 
T5 


Inadequate  Test  Case(s)  or  Test  Data 

Misinterpretation  of  Test  Results 

Mi s i n t eroret a t i on  of  Program  Specification 

Neq 1 i  gence 

Other  Testing  Errors 
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ANNEX  G 


Paqe  1  o'  5a 


PROJECT  *  : 
Program  oart 


COMPLEXITY  MEASURES: 

NUMBER  OF  STATEMENTS:  2   ■ 

MJfSSR  CF  NCCES:  2 
NUMBER  CF  ARCS  :  1 
NUfBER  CF  PATHS:  I 
CYCLCKATIC  iNUVeEP:   V(G)  = 


REACHABILITY  CF  NCCES: 
NCCE     1   :     1 
NGCE     2   :     1 

SUP:  2.CCOOOO 

REACHABILITY     INCEX 
CF    OIRECTEC    GRAPH: 


DIRECTED  GRAPH  REPRESENTATION 

CLEARIN,  CLEAROUT,  CAHOIN,  PRINTOUT, 
TYPIN,  TYPOUT,  FILE  I/O,  TERMINAL  I/O, 
GET  OLD  FILE  INFORMATION,  WHITE  N£l«  FILE 
INFORMATION,  CAR1,  CAR2,  CDR1,  CDR2, 
SETCAR1,  SETCAR2,  SETCDR1,  SETCDR2,  FREE1, 
FPEE2,  FINO  ID,  GETQVALUE,  ""A  In 


o 

0 


NOTE 


l.CCCOOC 
23  subroutines  have  the  same  structure. 
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ANNEX  G 


Pace  2    o+     ^^ 


DIS&CTEO  GRAPH  REPRESENTATION 
PROJECT  9    :   a 
Program  oart  :  ALLOCATE!,  ALLGCATE2,  CONTROL 


COMPLEXITY  MEASURES: 

NUMBER    OF    STATEMENTS:    8 
NUN8ER    CF    NCCcS:  ^ 

NUfeER    CF    ARCS     :  3 

r^BER    CF    PATHS:  2 

CYCLCMATIC    NUMcER:       V  (  G  )  ■ 


REACHABILITY  CF  NCCcS: 


NCCS  1  :  1 

ISCCE  2  :  1 

NCCS  3  :  1 

NCCE  4  :  1 


Sue:  4.CC0CC0 

Rc^Cr-ABILlTY     INDEX 
CF    CIPECTEC    GRAPr  : 


i.  :::oo3 
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AMNEx  G 


paqe  3  of  3-4 


OIHECTEO  G3APH  REPRESENTATION 
PROJECT  a  :   u 

Proqra-n  oar?  :  SPACE 


COMPLEXITY  MEASURES: 

NUMBER  UF  STATEMENTS:  a 

NUMBER  CF  NCCES:  4 

NUMBER  CF  ARCS  :  4 

.NUMBER  CF  PATHS:  2 

CYCLCy^TIC  NUMBER :  V  (  G )  = 


REACHABILITY  CF  NCCES 


NGCS 

NCCc 
NCCE 
iSCCE 


1 


SUN:  6.CC30JO 

REACHABILITY     INCEX 

CF    OIRECTEC    C-RAPF  : 


1.5CCOCO 
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ANNEX  G 


Pane  u    of     3a 


DIRECTED  GRAPH  REPRESENTATION 
PROJECT  a  :   a 
Program  aart  :  INITIALIZE!,  T  'i  T  T  T  AL  I  ZE2 


COMPLEXITY  '-'EASUWES: 

NUMBER  OF  STATEMENTS:   7  - 
NUMBER  Cc  NCCES:     5 
NU*BEP  Cf    ARCS  :     5 
MJCBEP  CF  PATHS:     2 
CYCLCMATIC  NUMBER:   V(G)  = 


REACHABILITY  CF  NCCES: 
NCCE  1  :  1 
NCCE  2  :  2 
NCCE  3  :  1 
NCCE  4  :  2 
N  Q  C  c     5   :     2 


SUM:      e.ccccco 

RcACrAE  IL  ITY  INCEX 
CF  OIRECTEC  GRJPh: 


5999 
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ANNEX  G 


P age  5  of  34 


DIRECTED  GPAPh  REPRESENTATION 
PROJECT  a    :       a 
Program  oart  :  INITIALIZE  iLL 


COMPLE) 

ITY 

'- 

'EASUSES: 

NUMBEP 

OF 

STATEMENTS:  56 

NUMBER 

CF 

NCCES 

:     9 

number 

CF 

ARCS 

:    11 

NU>6ER 

CF 

PATHS 

:     8 

CYCLCMATIC 

NUM6( 

IR:       V(G)  = 

REACHABILITY  CF 

NCCES: 

NCCE 

1 

1 

NCCE 

2 

2 

NOCE 

3 

1 

ISGCE 

4 

A 

NCCE 

e 

2 

NOCE 

c 

8 

NCCE 

7 

4 

NCCE 

a 

6 

NCCE 

<> 

8 

„„       SUP:  26.CCCCC 

REACHABILITY     INCEX 

Or    OIRSCTEC    C-R^PHJ:  4.222221 
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ANNEX  G 


faqe 


C  f      34 


0  I  PEC  r  El)  GRAPH  REPRESENTATION 


PROJECT  =  :   a 

P  roqr  am  Dart   :  E->0OR 


COMPLEXITY  MEASURES: 

NUMBER  OF  STATEMENTS:  2« 
NUfBER  CF  NCCES:     6 
NUNBE"  CF  ARCS  :     5 
JvUf-BER  CF  FATHS:     4 
CYCLCfATIC  NUMEER:   V(G)= 


REACHABILITY  CF  NCCES: 


NCCE 
NCCE 
NCCE 
iNCCE 
NCCE 
NCCE 


SUN:       6.CCCCCC 
S  E  ^CHA  5  IL  ITY  INCEX 
CF  CIPECTEC  GRAPH  : 


L.CCCOOO 
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AN.NEX  G 


Paoe  7  of  3a 


DIRECTED  GRAPH  REPRESENT  A  T  ION 
PROJECT  u    :       a 
Proaram  Dart  :  DIAGNOSE 


COMPLEXITY  MEASURES: 

NUMBER  OF  STATEMENTS:  83 
NUMBER  CF  NCCES:    16 
NUN6ER  CF  ARCS  :    23 
NUI^BER  CF  PATHS:    40 
CYCLCMATIC  NUM8ER :   V(G)= 


REACFAeiLIT'T  CF 


NCCE 
NCCE 
NOCE 
NOCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NGCE 
NOCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 


1 
2 

■a 

4 
5 
6 
7 

a 
s 

1C 

n 

12 
12 
14 
15 
lc 


NCDES: 

1 

1 


*•    Error    0     12 


1C 

c 

1C 
2C 

10 
2C 
2C 
2C 
2C 


SUC:       151.3000 
REAChAe  ILITY  INCEX 
CF  OIRECTEC  GRAPH: 


9.4275C3 
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ANNEX  G 


Pa< 


3  of  3J 


l)I»ECTED  GRAPH  REPRESENTATION 
PROJECT  a  :   4 
Program  Dart  :  EXTRACT,  FILL,  LISTS,  LIS^C,  LISTALL, 

FIND  VALUE  LENGTH 


COMPLEXITY  "EASUKES: 
.MUMPER  OF  STATEVEMTS: 


NUMBER  CF  \CCES:     5 
NUMBER  CF  ARCS  :     5 
NUMBER  CF  PATHS:     2 
CVCLCVATIC  NUMBER:   V  (  G  )  = 

REACHABILITY 
NCCE     I 
NCCE     2 
NCCe     3 
NCCE     A 
NCCE     5 

'  CF  NCCES: 
1 
2 
1 
2 
2 

SUN:       3.CCGC00 
REACHABILITY  INCEX 
CF  DIRECTED  GRAPH:       L . 5SS999 
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ANNEX  G 


Pace  <5  o*  3a 


DIRECTED  GRAPH  ^PRESENTATION 
PROJECT  a  :   a 
Program  Dart  :  GETCHAR 


COMPLEXITY  '-'E4SURES: 
NUMBER  OF  STATEMENTS:  21 


NUMBER 

CF 

NCCE 

::    12 

\UfS6R 

CF 

ARCS 

:    15 

MJ>6ER 

CF 

PATh 

::    16 

CYCLCMA7IC 

MIM8ER:   V< 

3EACHA8  IlITY  C. 

:  NCCES : 

\CCE 

1 

1 

NCCE 

2 

1 

NCCE 

3 

1 

NCCE 

4 

2 

NOCE 

c 

Z 

NCCE 

6 

14 

NCCE 

7 

12 

NCCE 

g 

c 

NCCE 

c 

12 

NCCE 

1G 

12 

NCCE 

11 

16 

NCCE 

12 

16 

)  = 


SUP:  95. cacao 

REACHAE  ILHY     INCEX 

CF    CIREC7EC    GRAPH:  7 
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ANHE<  G 


Paae  in  0f  3a 


OI^ECTtO  GRAPH  REPRESENTA T ION 
PROJECT  S  :   a 
Progra*  oart  :  SUPER  USER 


COMPLEXITY  MEASURES: 

NUM8ER  OF  STATEMENTS:  49 

NUMBER  CF  NCCES:  14 
NUMBER  CF  ARCS  :  17 
NUNBcP  CF  P4THS:  12 
CYCLCfATIC  NUMBER:   V(G)  = 


RE 
NC 

NC 

.\c 

NC 

No 
NC 
NC 
NC 
NC 
NC 

NC 
NC 
NC 
NC 


4CHA 

CE 

CE 

CS 

Cc 

CE 


CE 
CE 
CE 
CE 
CE 
CE 


BILITY 
1 
2 


6 
7 

S 

s 

1C 


CF 


11 
12 
12 
14 


nc; 
1 
1 
1 

2 
2 
2 

A 

£ 

E 

12 

12 


ES 


SUN:       65.C3COO 
REACHABILITY  INCEX 
CF  CIRECTEC  GRAPH  : 


4.642S57 
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ANNEX  G 


Pace  11 


ja 


DIRECTED  GRAPH  "  E  D  R  F  3  E \  T  A  T  I  0  N 
PROJECT  s  :   a 
Program  part  :  STORE 


COMPLEXITY  MEASURES: 

NUMBER  OF  STATEMENTS:  t<? 

NUfBEP  CF  NCCES:  7 
NUNBEP  CF  ARCS  :  3 
NUf-BER  CF  PATHS:  7 
CYCLC^ATIC  NUWEcB:   V(G)= 

REACHABILITY  CF  NCCES: 


NCCE 

1 

:     7 

NCCE 

2 

6 

NCCE 

"2 

- 

NCCE 

I 

:     5 

NCCE 

s 

6 

NCCE 

6 

6 

NCCE 

7 

7 

SUP:       3S.C3000 
SE^CHAcILITY  INCEX 
CF  DIRSCTEC  GRAPH: 


.557 
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ANNEX  G 


Paa?  12  of  34 


DIRECTED  GRAPH  REP»E SENT iTION 

PROJECT  a  :   a 

Proaram  oart  :  GET  ATTRIBUTES  AND  VALUES 


a )  a  o  f  nooes :  Q 

b )  a  0f  arcs  :  It 

c)  3    of    statements: 
a)  =  of  c a t * s :  * 

e)  "eacham  I  i  tyl  • 

f  )  CyClOT'atic  nufPer 
*  Siu'cer  of  oat»s 
ana  reachability 
ai*e  verv  1  a rce  . 


5  0 
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ANNEX  G 


13  of  3i» 


OIPECrED  GPuPH  PEPoFSE'-iTiT  ION 
PROJECT  =  :   a 
Proaram    cart     :  GET  ATTRIBUTE 


COMPLEXITY  MEASURES: 

NUMBER  OF  STATEMENTS:  23 

MJPBEP  CF  NCCES:  15 

NUfBER  CF  ARCS  :  18 

NUf-BEP  CF  FATKS:  16 

CYCLC,MATIC  NUNBER  :  V(G)  = 


sea 

NCC 
NCC 
NCC 
NCC 
NCC 
NCC 
NCC 
NCC 
NCC 
NCC 
NCC 
NGC 
NCC 
NCC 
NCC 


CFA 


eiLITY 
1 
2 


CF 


1C 
11 
12 
12 
LA 
15 


NCCES 

1 

1 

1 

1 


2 

M 

2 
t 
6 

12 
12 


Slif:  64.Q0J00 

REAChAE  II  ITV     INCcX 
CF    C  IRECTEC    C-RAPh  : 


<t  .26c666 
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ANNEX  G 


Paa<?  la  of  3a 


DIRECTED  GRAPH  REPRESENT ATIOM 
PROJECT  3  :   a 
Program  cart  :  GET  VALUE 


a)  *  of  nodes:  17 

b)  a  of  arcs  :  23 

c)  -    of  stat  e^oit s : 
n)     a  of  oatris:  ♦ 

?)  ^eac  ">  ac  i  '  i  t  v  :  * 
f)  Cyc'OTiatic  n.jncer 
*  N  o  m  o  e  r  of  oaths 
ana  reachability 
are    verv  I  a  roe  . 


Error     3  1  ■'■! 


*■  Error  =  15 


Error     =  to 
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ANNEX  G 


Fane  15 


54 


DIRECTED  Grjph  REPRESENTATION 
PROJECT  fi  :   a 
Proaram  oart  :  F I .'40  ATTRIBUTE 


COMPLEXITY  MEASURES: 

MUMBER  OF  STATEMENTS:  13 

NUMBER  CF  NCCES:  6 
NUMBER  CF  ARCS  : 

NUMBER  CF  FATKS:  4 

CYCLCMATIC  MJfBER  :  V (G ) 

REACl-AeillTY  CF  NQCSS: 
1 
2 
1 
2 
2 
4 


«-     Error 


NCCE 

I 

NCCc 

2 

NCCE 

NCCE 

A 

NCCE 

c 

NCCE 

6 

SUf:  12.CC000 

REACrAeil  IT>     INCS.X 
CF    DIP.  EC-TEC    GPAPH: 


;ccoo 


2  0 
lb 
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ANNEX  G 


Peae  lo  of  3 a 


DIRECTED  GRAPH  REPRESENTATION 
PROJECT  s  :   a 
Program  oart  :  SKIP,  FIN!)  COfMAwD',  GET  CONDITIONS, 

TEST  iLL 


COMPLEXlTr  MEASURES: 

NUMBER  OF  STATEMENTS:  b    - 

,\UN8EP    CF    NCCES:  4 

NUMBER    CF    ARCS     :  4 

iNUfocR    GF    =ATKS:  2 

CYCIC*ATIC    NUPEEfl:       V(G)  = 


3E*CJ-AE  ILITY    CF    NCCES 


NGCE 

1   : 

1 

NCC5 

I 

2 

NGCE 

3   : 

1 

NGCE 

4   : 

2 

SUf:  fc.CCGCCO 

REACh^E  IL  !TY     INCEX 
CF    DIRECTEC    GRAPH  : 


12 


1.5CC0CJ 
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ANNEX  G 


Pace  17  o  f     3a 


DIRECTED  GRAPH  PEPRESEN  TA  T  I  flN 
PROJECT  s  :   u 
Program  oart  :  INTERPRET 


COMPLEXITY  ^EASUHES: 


NUMBER  OF  STATEMENTS: 

MJf"6ER  CF  NCCES: 
MiCeEP  CF  ARCS  : 
NUfSER  CF  FAThS: 
CVCLC^ATIC  NC^eEF 

V(G>  =      1 

rASIUTY  CF 
I 


REAC 
NCCE 
NCCE 
NCCE 
NCCE 
NGCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 


c 
3 

6 
7 

8 
9 
1C 
11 
12 
13 
14 
15 
16 
17 
IS 


13 
17 
16 


CCE5 


Sit*:  1€.CCCCC 

REACHABILITY  INCEX 
CF  CIS  EC  TEC  GRJPr  : 


l.CCCCOO 
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ANNEX  G 


Paa<?  18  of  3a 


PROJECT  3  : 
P  rograc  can 


DIRECTED  g^aph  weppESEuTATIOn 


SHIFT 


COMPLEXITY  '"EASUHE3: 

NUMBER  OF  STATEMENTS:  ,21 
NUf-BEP  CF  NCCES:    11 
NUfBEP  CF  A3CS  :    13 
NUI^BSP  CF  PATHS:     3 
CYCLC.^ATIC  NUMBER:   V(G)  = 


5EACI-A 

EILITY    CF 

NCOES: 

NCCE 

1 

1 

NCCE 

2 

I 

NCCE 

i 

i 

NuCE 

I 

1 

NCCE 

5 

<j 

NCCE 

6 

2 

NCCE 

7 

4 

NCCE 

6 

£ 

ncce 

5 

4 

NCCE 

1C 

I 

NCCE 

11 
1* : 

a 

SL 

4 

I . ccccc 

^E^CHAB  IL  IT> 

'     INC  EX 

CF    DIP. 

ECTEC 

CRAPI-  : 

3.618181 
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ANMEX  G 


Pacp 


of  3j 


DIKECTFO  GRAPH  REPRESENTATION 
PROJECT  =  :   a 
Program  oarr  :  LISTD^ASE 


COMPLEXITY  MEASURES: 

NUMBER  CF  STATEMENTS: 

NUfBEB  CF  NCCES: 
NUMBER  CF  ARCS  : 
MjfBER  CF  PATHS: 
CYCLCVATIC  NUV8EF 


1  1 

6 
7 

V(  Gh 


REACHABILITY 

NGCE  I 

NCCE  I 

NCCS  3 

l\CCS  A 

NCCE  5 

NGCE  c 


CF 


NCC! 

1 

1 

M 

2 


SLP:  16.C0CCC 

REACHABILITY     INCEX 

CF    OIRECTEC    GRAPH  : 


2 . £66566 
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ANi\EX    G 


P aaa    iO    of     34 


DIRECTED  GRAPH  REPRESENTATION 
PROJECT  s  :  n 
Program  cart  :  LISTSHCRT 


COMPLEXITY  MEASURES: 

NUMBER  OF  STATEMENTS:  13 

NUMBER  CF  NCCES:  3 
NUMBER  CF  ARCS  :  8 
.NUrBER  CF  PATHS:  3 
CYClCVATIC  NUMBER:   V(G)= 

REACHABILITY  CF  NCCES: 


\CCb 

1 

VCCE 

2 

\CCz 

"3 

NCCc 

A 

*ZC£ 

5 

NCCE 

6 

NCCE 

7 

NCCE 

a 

SUP:       9.C00CC0 
P.  E^C  hi  e  IL  ITY  INCEX 
CF  DIRECTEC  GRAPH  : 


i.l25O0C 
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ANNEX  G 


Pace  21  c  *  54 


DIRECTED  GwAPn  »EPREStMATIOi'j 
PROJECT  «  :   a 
Program  cart  :  FTMO  "EMBER,  FINQAT  TR I6UTE ,  ivRIFE  RESULTS 


COMPLEXITY  '"EASURES: 

MUMRE0  OF  STATEMENTS:  13  -  !o 
NUPBER  CF  NCCES:     7 
NUCBER  CF  ARCS  :     3 
MJ>8Eh  CF  PATHS:     7 
CYCLCPATIC  NUMBER:   V(G)=     ■ 


REACI-ABIL  ITY  CF 


NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 


NCCE. 
1 

7 


SXlf.  34.CCCC0 

SEACI-AE  IL  ITV     INCcX 
CF    CIRECTEC    G?*P\-  : 


4. £57142 


a    2«5       (in 

R e  s  u 1 ts") 


-    298    - 


ANME.X  R 


Pace  2  2    of  i-i 


CIRFCTED  1*19*    =EpPF5t'jTAT  IG: 
PROJECT  a  :  a 
Program  cart  :  L ISJ ATTRIBUTE 


a)  3  of  ncTes:  16 

b)  a  of  arcs  :  2 1 

c)  =  of  sta'»T»its:  3  'J 

d )  a  of  c^r^s:  » 

e )  °eac "ability:  * 

f  )  Cyc  loint  ic  n  u  m  n  e  r  :  7 
«  \uTer  of  aat^s 

and  r e  ac  ^  an i 1 i  t  v 

are    v»ry  la  roe  . 
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ANNEX  G 


Pwe  21    of  5u 


.  DIRECTED  GRAPH  REPRESENT  A T 10! 
PROJECT  a  :   a 
Program  cart  :  LIST'-' 


COMPLEXITY  MEASURES: 

NUMBER  OF  3TATEMEMT3:  2« 
NUMBER  CF  NCCES:    13 
\Uf6ER  CF  ARCS  :    16 
NU>BER  CF  PATHS:    13 
CYCLCMATIC  NURSES:   V(G)= 


RE4C 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCt 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 


FA6IL  ITY 
1 
2 


£ 

7 
£ 
S 
1C 
11 
12 
12 


CF 


NCCES 

1 

1 

2 

2 

2 

2 

2 

A 

6 

a 

16 


SLf:  6C.CCC! 

PE^CHAeiLITV     INCcX 
CF    CIRECTEC    GR4PI-  : 


4.615  334 


E  r  r  q  r  \  ~     P 4 
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Af'lMEX     fi 


Pace    aa    of     ia 


PROJECT    a     : 

P  roQ ran    oart 


DIRECTED    GRAPH    n>EPPESEi')T  A  T  I  Of 
COMPARE    RANGE 


COMPLEXITY     vcflSUkES. 


19 


MUM8ER  OF  STATEMENTS: 

Mlf-BEF  CF  NCCES:  1<5 
Mil"  SEP.  CF  ARCS  :  24 
NUl*8ER  CF  PATHS:  16 
CYCLCMATIC  NOMEER:   V(G)= 


REAOAeiLITI 

'  CF  NCCES 

NCCE 

1 

1 

NCCE 

2 

1 

NCCE 

i 

1 

NCCE 

4 

I 

NCCE 

c 

1 

NCCE 

6 

1 

NGCE 

7 

1 

NCCE 

^ 

■     2 

NCCE 

S 

2 

NCCE 

LC 

4 

NCCE 

11 

4 

NCCE 

12 

4 

NCCE 

12 

4 

NCCE 

14 

4 

NCCE 

15 

u 

NCCE 

16 

c 

NCCE 

17 

6 

NCCE 

ia 

ie 

NCCE 

l? 

:    16 

SUf:       32. COJJ: 
REACHAeiL  ITV  INCEX 
CF  DISECTEC  GRAPH  : 


4.2684 
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ANNEX  G 


Pane  2^  o*  3a 


DIRECTED  GRAPH  REPRESENTATION 
PROJECT  3  :   a 
Program  oar?  :  ATTR,  VATCHINO,  CHANGE  *"Er 


COMPLEXITY    " 

-'EASURES: 

MUMBER    OF    STATEMENTS: 

NUNBER    CF    NCCES:             6 
MJNBER    CF    ARCS     :             6 

NUMBER    GF    PAThS:            2 
CYCLCr'ATIC    NUMBER:       V( 

RE^CI-ABUm 
,NCCE             1 
NGCE             2 
fsCCE            2 
NGCE             A 
\CZc             5 
NGCE            6 

'    CF    \CCE5: 

1 

1 
:            I 

1 

2 

2 

10  -  17 


)  = 


SUM:  8 . C  C  C  3  0  0 

REACbAE  II.  ITY     INCE  X 
CF    OIRECTEC    GRAPH: 


1.2223: 
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ANNEX  G 


Pace  2b    of  3J 


OIRECTFD  GRAPH  REPRE3EN  T  A  T  I  ON 
PROJECT  s  :   a 
P  roar  a^    03 r t  :  F A 


COMPLEXITY  MEASURES: 

NUMBER  OF  STATEMENTS:  l^ 

NUfBER  CF  NCCES:  10 
NURSES,  CF  ARCS  :  11 
NUrBER  CF  PATHS:  2 
CYCLCMATIC  M-fBE*  :   V(G)  = 

SEACrABlLlTV  CF  NCCES: 


NCCE 

1 

NCCE 

2 

NCCE 

3 

iNCCE 

4 

NCCE 

5 

NCCE 

6 

NCCE 

7 

NCCE 

8 

NCCE 

<S 

NCCE 

ia 

SUV:  15.C000O 

5EACI-AS  IL  ITY     INCEX 
CF    OIRECTEC    GRAPr  : 


1.5CC0C0 
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ANNEX  G 


Paae  27    of  3a 


DIRECTED  GRaPh  REPRESENTATION 
PROJECT  2  :  n 
Program  oarr  :  FIND 


COMPLEXITY 

UEASUWES: 

NUMBER 

OF 

STATE 

1ENTS:  18 

MJC  BEfi 

CF 

NCC'ES 

:    11 

MlCBES 

CF 

ARCS 

:    li 

NURSES 

CF 

FATI-S 

:     5 

CVCLCNATIC 

NfPB 

E=  :   V(G)  = 

REACHABILITY  CF 

N  C  C  E  S : 

\CCt 

1 

1 

NCCE 

2 

1 

NCCE 

■3 

1 

NCCE 

H 

2 

\CCz 

B 

1 

iNCCE 

& 

2 

VCCE 

7 

2 

\£CE 

8 

2 

\OCE 

S 

2 

NCCE 

IC 

A 

NGCE 

11 

c 

SUf:  22.CCCGO 

sEAChAEILlTV     INCEX 
CF    CIRECTEC    GFAFr  : 


2  .C9C9C9 


s    ?5 


=    32 
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ANNEX     G 


Facie    26    of     34 


ore>EcrE'">  hpiph  representation 

PROJECT  a     :  Q 

Program  osrf  :  GET  REQUEST 

a  )  -of  nodes :  1 2 

b  )  a  of  arcs  :  IS 

c)  3  of  stat  e"e" t s :  35 

d)  a  of  oar^s:  * 

e )  Reachability;  * 

f)  CvClOTariC  n  ' j  t.  o  e  r  : 
*  N  umber  of  satns 

ano  reacnaoi  1  I  tv 
are  vesv  1  a  r  j  e  . 
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ANUEX  G 


'ace  2a  o  t  3-i 


DIRECTED  GRaPH  REPRESENTATION 
PROJECT  s  :   a 
°roqram  oart  :  E'3i;AL 


COMPLEXIT' 


NUMB 
NUfE 
NUfB 

Ncr^e 

CYCL 


«E4SU*E 


3  =  JC 


NCCE 

1    ! 

1        ^vT  ^ 

NCCE 

2   • 

1          ^v  ^S 

NCCE 

a    ; 

1            \ 

NCCE 

4   ' 

1             \ 

NCCE 

£ 

2 

NCCE 

6 

1 

l\CCE 

7   : 

2 

NCCE 

6   ■ 

2 

NCCE 

9 

L4 

NCCE 

1C 

12 

NGCc 

11 

6 

NCCE 

12 

12 

NCCE 

13 

12 

NCCE 

14 

14 

NCCE 

15 

:    14 

NCCE 

16 

:    2§ 

NOCE 

17 

i    2  2 

NCCE 

17 

:     C 

NCCE 

IS 

:    22 

NCCE 

IS 

64 

NCCE 

2C 

:    e4  - 

NCCE 

21 

:    64 

NGCE 

22 

123 

NCCE 

■>  ' 

:   IcC 

NCCE 

24 

:   16C 

NCCE 

25 

:   223 

NCCE 

26 

:       321 

SOf  : 

1468.000 

SEiChA 

eiLiT' 

1     INCEX 

CF  C  IR 

ECTEC 

GR4PI-:       54.3703c 
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ANNEX  G 


Pace  30  of  5a 


DIRECTED  graph  oE°WESE'i  T  AT  ION 
PROJECT  3  :   u 
Program  oart  :  TEST  CONDITIONS 


COMPLEXITY  -"ElSUKES: 

NUMBER  OF  STATEMENTS:  19 

M^8ER  CF  NCCES:  1J 
NUMBER  CF  ARCS  :  12 
MJfrBER  CF  PATH  S  :  6 
CYCLCfATIC  NUMBER:   V(G)  = 


RE^ChiSlLlTV  CF  NCCcS 


INGCs 
NCC£ 

f^CCz 
NCC5 
NCCE 
.\CCE 
NCCE 
NCC5 
NCCE 
iNCCE 


6 
7 
S 
9 
1C 


SUf:  24.C0OO0 

'EAChAEILITY     INCEX 
CF    DIRECTEC    GR4PI-: 


2.4CCCC0 


26 
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ANNEX  G 


Page  31  of  3a 


DIRECTED  GRAPH  REPRESENTATION! 
PROJECT  u    :   a 
Program  cart  :  COMPARE  MANY 

a )  a  of  no<ies  :  1  2 

b)  «  of  arcs  :  la 

c)  *    of  statements:  21 

d)  a  of  oaths:  * 

e )  Reac  nab  i I i t v  :  * 

f)  Cyclomatic  number:  a 
*  Numbe  r  of  oaths 

and  reac  hab  i  1 i  t  v 
are    very  I aroe . 
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4.vjne<   o 


Pace    32    of     3^ 


DIRECTED  RniPh  REPRfcSEM TAT  ION 

PROJECT  s  :   a 

Proa ram  oart  :  L L  I ■  ^  I T 


COMPLEXITY  "'ENSURES: 

DUMBER  OF  STATEMENTS: 

NUMBER  CF  NCCES:     7 

NUMBER  CF  ARCS  :     d 

NUrSSR  CF  FAThS:     7 


15 


CYCICVATIC  NUMBER 


v(C-  )  = 


RE4CHA 

Bit 

[TY  CF  NC05S 

NOCS 

1 

1 

NCCE 

2 

7 

NGCE 

3 

,     f 

NCCE 

4 

~ 

NCCE 

e 

c 

NCCE 

6 

7 

NCCE 

7 

1 

Sty:       26. C0C0O 
REACHABILITY  INCE.X 
CF  DIRECTEC  GRAPh  : 


,42357 
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A  M  ivj  £  X  G 


Pace  33  of  3  -i 


DIRECTED  RPiPH  REPRESENTATION 
PROJECT  a  :   a 
Program  oart  :  ULIMIT 


COMPLEXITY  MEASURES: 

NUMBER  OF  STATE 

MfJTS:  17 

NUMBER  CF  NCCES 

:    IC 

NUMBER  CF  ARCS 

:    12 

NUf-BES  CF  PATHS 

:    1* 

CYCLCMATIC  NUME 

EB:   V(G )= 

REACHABILITY  CF 

NCCES : 

NCCE     1 

I 

NCCE     2 

2 

NCCE     i 

1 

NCCE     4 

2 

NCCE     5 

14 

NCCE     t 

IC 

NCCE     7 

6 

NCCE     6 

12 

NCCE     <5 

14 

NCCE    IG 

14 

SUM:  76.CCCC0 

REACHABILITY     INCtX 
CF    DIR.ECTEC    GRAPH: 


T.fssgss 
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ANNEX  G 


Paqe  3u  of     3d 


OI»ECTEO  GRAPH  ^PRESENTATION 
PROJECT  =  :   a 
Program  oarr  :  SWITCH 


COMPLEXITY 

MEASURES 

: 

NUMBER* 

OF 

STATEMENT 

5 : 

15 

NUMBER  CF 
NUMBER  CF 
MjNBEfi  CF 
CYCLCMATIC 

NCCES: 
iRCS  : 
PATHS: 

NUMBER  : 

4 

2 

V  (  G  )  = 

REACHABILITY  CF 
NCCE     1   : 
SCCE     2   : 
NCCS     3   : 
NGCE     4   : 

NCO 
1 
1 
1 

2 

ES: 

SLM:  5.CC0OCO 

REACHABILITY     INCEX 
CF    CIRECTEC    GRAPH: 


1  .23COOC 
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ANNEX  H 


Page  I   of  3 


TEST  PHASE  DESCRIPTION 
Project  *  :  4 

Test  run  #  :  1         Including  5   Test  SteDS 
Begin  of  Test  (day/time)  :  Ou/29/1330     End  of  Test  (day/time)  :  0^/29/1530 


CPU  time  for  necessary   compiles  (in  sec.):  9.09 
a)   9.09   b)        c)        d)       e)         f) 


g) 


CPU  time  for  TEST  run  (sec) 


12.8 


Man   Hours  for  this  Test  run  :  2.0  (including  Dreoaration  of 
tests) 


TEST 
STEP 


OBJECTvE 


EXPECTED 

RESULT 

(TOLERANCE) 


1) 
ACTUAL1ERR0R 
RESULT!  * 


DAY 
TIME 


COMMENTS 

AND 
COOED  ERROR 
TYPES 


Test  of  I/O  func- 
ions  unaer  CP/CMS 


Check  Boundary 
conditions  under 
CP/CMS 

Test  SU  fund  i  ons 
under  CP/CMS 


Check  Di  agnos t  i  cs 
Test  Termination 
O  f  progr am 


I/O 

ted 

-Oat 

mat  i 

f  rom 

"DBA 

-Com 

i  nou 

na  1 

-Out 

di  re 

to  t 

file 

Prog 

i  npu 

aoor 

mess 

a)Me 

adde 

base 

b)0a 

form 

can 

outo 

that 

coul 

i  npu 

subs 

c)It 

be  e 


can  be 
as  desi 
a  base 
on  i  s  r 
file  1 
SE  INPU 
manas  a 
t  y  i  a  t 
on  1  y 
put  can 
c  t ed  by 
erm  i  na 1 

ram  neg 
t  or  or 
oori ate 
ages 

mbers  c 
d  to  t  h 

t a  base 
at  i  on  c 
be  writ 
ut  file 

this  f 
d  be  us 

t  file 
eauent 

ems  o  f 
x  am  i  nea 


di  rec- 
qned : 
i  n  f  or- 
ead 

abe 1 ed 
T" 
re 
ermi  - 

be 

user 

or 

1  ect  s 
ov  i  des 
error 

an  be 
e  data 

i  n- 
an  be 
ten  on 

such 
i  le 
ed  as 
for 
runs 
M2  can 


O.K. 


O.K. 


O.K 


O.K 
O.K 


a/29 
1330 


u/29 
1550 


1  )  Record  when 
error  occurs 
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ANNEX  H 


Page  2   of  3 


TEST  PHASE  DESCRIPTION 
Project  *    :  <J 

Test  run  #  :  2         Including  2   Test  Steps 
Begin  of  Test  (day/time)  :  05/01/1200     End  of  Test  (day/time)  :  05/02/2200 


CPU  time  for  necessary   compiles  (in  sec):  8.75 
a)   8.75   b)        c)        d)       e)         f) 

CPU  time  for  TEST  run  (sec)  :  12.40 


9) 


Man  Hours  for  this  Test  run  :  U.O  (including  oreparation   of 
tests) 

1) 


TEST 

08JECTVE 

EXPECTED 

ACTUAL 

ERROR 

DAY 

COMMENTS 

STEP 

RESULT 
(TOLERANCE) 

RESULT 

a 

TIME 

ANO 
CODED  ERROR 
TYPES 

1 

Test  comoar i son 

Comoarison  must 

Does 

5/01 

1)  Record  when 

of  range  values 

always  be  correct 

not 
work 
for 
nega- 
t  i  ve 
num- 
bers 

1200 
1700 

5/01 
1730 

error  occurs 
03 

I  mo  1 emen  t  a  t  i  on 
of  new  coae 
necessary  t  o 
handl e  all 
cases 

2 

Reoeat  step  1 
us  i  ng  cor rec t  ed 
version  of  program 

O.K. 

5/02 
1600 
5/02 
2200 
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ANNEX  H 


Page  3   of  3 


TEST  PHASE  DESCRIPTION 
Project  *  :  « 

Test  run  tt    I     3         Including  1   Test  Steps 
Begin  of  Test  (day/time)  :  05/03/0900     End  of  Test  (day/time)  :  05/03/1900 


CPU  time  for  necessary   compiles  (in  sec.):  9.68 
a)   9.68   b)        c)        d)       e)         f) 

CPU  time  for  TEST  run  (sec)  :  12. 81 


g) 


Van   Hours  for  this  Test  run  :  5.0  (including  oreoaration  of 
tests) 


1) 

TEST!      0BJECTVE 

EXPECTED 

ACTUAL 

ERROR 

DAY 

COMMENTS 

STEP! 

PESULT 
(TOLERANCE) 

RESULT 

tt 

TIME 

AND 
CODED  ERROR 
TYPES 

1   ITest  oroaram  unaer 

A  1  )  func t  i  ens 

5/03 

1  )  Record  when 

ICP/C^S  using 

shou Id  xork  the 

0Q00 

error    occurs 

! radars  as  data 

same  way  as  for 

O.K. 

J  members 

members  used  in 
orev  i  ous  test  and 
neoug  runs 
(opov i  ded  that 
a^     attribute 
va 1 ue  oai  rs  are 
within  des  i  gned 
1  i  mi  t  S  ) 

5/03 
1800 
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